2024-09-10T18:09:39.243 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:09:39.258 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:09:39.332 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7900216' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:09:39.332 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:09:39.333 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:09:39.333 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:09:39.335 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:09:39.356 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:09:39.356 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:09:39.356 INFO:teuthology.packaging:ref: None 2024-09-10T18:09:39.356 INFO:teuthology.packaging:tag: None 2024-09-10T18:09:39.356 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:09:39.356 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:09:39.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:09:39.494 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606-g5be57395-1focal 2024-09-10T18:09:39.496 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:09:39.505 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:09:39.505 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:09:39.520 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:09:39.532 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:09:39.543 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:09:39.568 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:03:30.093656', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-10T18:09:39.589 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:03:30.095473', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-10T18:09:39.589 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:09:39.598 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:09:39.598 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:09:39.598 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:09:39.660 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc3038bbac0>, signals=[15]) 2024-09-10T18:09:39.660 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:09:39.672 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:09:39.672 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-09-10T18:09:39.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:09:39.769 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-09-10T18:09:39.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:09:39.856 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:09:39.868 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-09-10T18:09:39.874 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-09-10T18:09:39.875 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-09-10T18:09:39.923 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Ubuntu" 2024-09-10T18:09:39.923 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-10T18:09:39.923 INFO:teuthology.orchestra.run.smithi100.stdout:ID=ubuntu 2024-09-10T18:09:39.923 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE=debian 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="20.04" 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_CODENAME=focal 2024-09-10T18:09:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:UBUNTU_CODENAME=focal 2024-09-10T18:09:39.925 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-09-10T18:09:39.945 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-09-10T18:09:39.952 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-09-10T18:09:39.952 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-09-10T18:09:40.005 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Ubuntu" 2024-09-10T18:09:40.005 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-10T18:09:40.005 INFO:teuthology.orchestra.run.smithi151.stdout:ID=ubuntu 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE=debian 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="20.04" 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_CODENAME=focal 2024-09-10T18:09:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:UBUNTU_CODENAME=focal 2024-09-10T18:09:40.006 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-09-10T18:09:40.028 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:09:40.042 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:09:40.055 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:09:40.055 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:09:40.057 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:09:40.062 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:09:40.074 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:09:40.074 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:09:40.103 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:09:40.121 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:09:40.180 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:09:40.199 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:09:40.200 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T18:09:40.200 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:09:40.200 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:09:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:09:40.206 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-10T18:09:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:09:40.207 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-09-10T18:09:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:5.4.0-124-generic 2024-09-10T18:09:40.213 INFO:teuthology.task.kernel:Running kernel on smithi151: 5.4.0-124-generic 2024-09-10T18:09:40.213 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get clean 2024-09-10T18:09:40.255 INFO:teuthology.orchestra.run.smithi100.stdout:5.4.0-124-generic 2024-09-10T18:09:40.256 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.4.0-124-generic 2024-09-10T18:09:40.256 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-09-10T18:09:40.366 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-09-10T18:09:40.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-09-10T18:09:40.529 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-10T18:09:40.639 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-10T18:09:40.679 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-10T18:09:40.679 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-10T18:09:40.721 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-10T18:09:40.738 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-10T18:09:40.826 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,166 kB] 2024-09-10T18:09:40.942 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [807 kB] 2024-09-10T18:09:40.952 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [469 kB] 2024-09-10T18:09:40.957 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-10T18:09:40.957 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-10T18:09:40.958 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,082 kB] 2024-09-10T18:09:40.979 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [431 kB] 2024-09-10T18:09:40.982 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-10T18:09:40.982 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [680 kB] 2024-09-10T18:09:40.986 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,009 kB] 2024-09-10T18:09:40.993 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-10T18:09:40.994 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-10T18:09:40.994 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-10T18:09:40.994 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-10T18:09:41.007 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-10T18:09:41.007 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-10T18:09:41.073 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-10T18:09:41.091 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-10T18:09:41.173 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,166 kB] 2024-09-10T18:09:41.214 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,027 kB] 2024-09-10T18:09:41.314 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,550 kB] 2024-09-10T18:09:41.457 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,550 kB] 2024-09-10T18:09:41.625 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [548 kB] 2024-09-10T18:09:41.637 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-10T18:09:41.638 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,215 kB] 2024-09-10T18:09:41.678 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [807 kB] 2024-09-10T18:09:41.691 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [469 kB] 2024-09-10T18:09:41.700 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-10T18:09:41.700 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-10T18:09:41.701 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,082 kB] 2024-09-10T18:09:41.731 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-10T18:09:41.732 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [450 kB] 2024-09-10T18:09:41.749 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-10T18:09:41.749 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-10T18:09:41.779 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,027 kB] 2024-09-10T18:09:41.785 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-10T18:09:41.789 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [431 kB] 2024-09-10T18:09:41.796 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [548 kB] 2024-09-10T18:09:41.796 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-10T18:09:41.796 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,009 kB] 2024-09-10T18:09:41.804 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-10T18:09:41.804 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-10T18:09:41.805 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-10T18:09:41.805 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,215 kB] 2024-09-10T18:09:41.812 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-10T18:09:41.812 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-10T18:09:41.813 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-10T18:09:41.846 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [680 kB] 2024-09-10T18:09:41.858 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-10T18:09:41.861 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-10T18:09:41.862 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-10T18:09:41.862 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-10T18:09:41.862 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-10T18:09:41.871 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-10T18:09:41.872 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-10T18:09:41.872 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-10T18:09:41.873 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-10T18:09:41.874 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-10T18:09:41.895 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [450 kB] 2024-09-10T18:09:41.902 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-10T18:09:41.902 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-10T18:09:41.936 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-10T18:09:41.953 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-10T18:09:41.953 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-10T18:09:41.954 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-10T18:09:41.958 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-10T18:09:41.972 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-10T18:09:41.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-10T18:09:41.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-10T18:09:41.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-10T18:09:41.977 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-10T18:09:42.042 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-10T18:09:42.042 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-10T18:09:42.043 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-10T18:09:42.044 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-10T18:09:42.123 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-10T18:09:42.124 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-10T18:09:42.124 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-10T18:09:45.256 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 21.7 MB in 3s (7,446 kB/s) 2024-09-10T18:09:45.933 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 21.7 MB in 4s (6,042 kB/s) 2024-09-10T18:09:46.854 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-09-10T18:09:46.882 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-10T18:09:46.975 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-09-10T18:09:47.194 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-09-10T18:09:47.194 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-09-10T18:09:47.406 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-09-10T18:09:47.406 INFO:teuthology.orchestra.run.smithi151.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-10T18:09:47.406 INFO:teuthology.orchestra.run.smithi151.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-10T18:09:47.406 INFO:teuthology.orchestra.run.smithi151.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-10T18:09:47.407 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-09-10T18:09:47.407 INFO:teuthology.orchestra.run.smithi151.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-10T18:09:47.407 INFO:teuthology.orchestra.run.smithi151.stdout:Recommended packages: 2024-09-10T18:09:47.408 INFO:teuthology.orchestra.run.smithi151.stdout: thermald 2024-09-10T18:09:47.408 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-10T18:09:47.436 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-10T18:09:47.459 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-09-10T18:09:47.459 INFO:teuthology.orchestra.run.smithi151.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-10T18:09:47.459 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-10T18:09:47.459 INFO:teuthology.orchestra.run.smithi151.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-10T18:09:47.461 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages will be upgraded: 2024-09-10T18:09:47.461 INFO:teuthology.orchestra.run.smithi151.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-10T18:09:47.506 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-10T18:09:47.517 INFO:teuthology.orchestra.run.smithi151.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-10T18:09:47.517 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 77.2 MB of archives. 2024-09-10T18:09:47.517 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-10T18:09:47.517 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-10T18:09:47.724 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-09-10T18:09:47.725 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-09-10T18:09:47.850 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-10T18:09:47.931 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-09-10T18:09:47.931 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-10T18:09:47.932 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-10T18:09:47.932 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-10T18:09:47.933 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-09-10T18:09:47.933 INFO:teuthology.orchestra.run.smithi100.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-10T18:09:47.933 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-09-10T18:09:47.933 INFO:teuthology.orchestra.run.smithi100.stdout: thermald 2024-09-10T18:09:47.943 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-10T18:09:47.977 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-09-10T18:09:47.977 INFO:teuthology.orchestra.run.smithi100.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-10T18:09:47.977 INFO:teuthology.orchestra.run.smithi100.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-10T18:09:47.977 INFO:teuthology.orchestra.run.smithi100.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-10T18:09:47.979 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-09-10T18:09:47.979 INFO:teuthology.orchestra.run.smithi100.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-10T18:09:48.032 INFO:teuthology.orchestra.run.smithi100.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-10T18:09:48.033 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 77.2 MB of archives. 2024-09-10T18:09:48.033 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-10T18:09:48.033 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-10T18:09:48.364 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-10T18:09:48.404 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-10T18:09:48.404 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-10T18:09:48.404 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-10T18:09:48.457 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-10T18:09:48.498 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-10T18:09:48.509 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-10T18:09:48.946 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-10T18:09:48.946 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-10T18:09:48.990 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-10T18:09:49.088 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-10T18:09:49.098 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-10T18:09:49.194 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 77.2 MB in 1s (74.2 MB/s) 2024-09-10T18:09:49.384 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-10T18:09:49.830 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 77.2 MB in 1s (69.4 MB/s) 2024-09-10T18:09:50.176 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-10T18:09:51.121 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-10T18:09:51.126 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:09:51.225 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:09:51.909 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-10T18:09:51.914 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:09:52.007 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:09:53.213 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-10T18:09:53.229 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:09:53.321 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:09:53.915 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-10T18:09:53.930 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:09:54.031 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:09:54.400 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-10T18:09:54.415 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:09:54.450 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:09:55.126 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-10T18:09:55.141 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:09:55.184 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:09:59.369 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-10T18:09:59.561 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-10T18:09:59.879 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-10T18:09:59.930 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-10T18:10:00.007 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-10T18:10:00.080 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-10T18:10:00.354 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-10T18:10:00.368 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-10T18:10:00.398 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-10T18:10:00.412 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-10T18:10:00.525 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-10T18:10:00.895 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-10T18:10:00.912 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-10T18:10:00.953 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-10T18:10:06.219 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-10T18:10:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:10:06.286 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:06.571 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-10T18:10:06.594 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-10T18:10:06.637 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:09.037 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-10T18:10:09.164 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-10T18:10:09.301 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-10T18:10:09.423 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-10T18:10:09.487 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:09.773 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:10.749 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:10.988 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:11.899 INFO:teuthology.orchestra.run.smithi151.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-10T18:10:11.899 INFO:teuthology.orchestra.run.smithi151.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:12.026 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:12.140 INFO:teuthology.orchestra.run.smithi100.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-10T18:10:12.140 INFO:teuthology.orchestra.run.smithi100.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:12.281 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:13.450 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-10T18:10:13.609 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-10T18:10:13.653 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-10T18:10:13.761 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:13.796 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-10T18:10:13.888 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-10T18:10:13.923 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:14.013 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-10T18:10:14.049 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-10T18:10:14.164 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:14.175 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-10T18:10:14.237 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-10T18:10:14.238 INFO:teuthology.orchestra.run.smithi100.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:14.302 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-10T18:10:14.370 INFO:teuthology.orchestra.run.smithi151.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-10T18:10:14.370 INFO:teuthology.orchestra.run.smithi151.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:36.235 INFO:teuthology.orchestra.run.smithi151.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-10T18:10:36.235 INFO:teuthology.orchestra.run.smithi151.stdout:Sourcing file `/etc/default/grub' 2024-09-10T18:10:36.248 INFO:teuthology.orchestra.run.smithi151.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T18:10:36.255 INFO:teuthology.orchestra.run.smithi151.stdout:Generating grub configuration file ... 2024-09-10T18:10:36.630 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-10T18:10:36.650 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:36.875 INFO:teuthology.orchestra.run.smithi100.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-10T18:10:36.876 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub' 2024-09-10T18:10:36.890 INFO:teuthology.orchestra.run.smithi100.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T18:10:36.905 INFO:teuthology.orchestra.run.smithi100.stdout:Generating grub configuration file ... 2024-09-10T18:10:37.021 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-10T18:10:37.031 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-10T18:10:37.225 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-10T18:10:37.245 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:37.643 INFO:teuthology.orchestra.run.smithi100.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-10T18:10:37.653 INFO:teuthology.orchestra.run.smithi100.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-10T18:10:39.189 INFO:teuthology.orchestra.run.smithi151.stdout:done 2024-09-10T18:10:39.613 DEBUG:teuthology.orchestra.run.smithi151:> dpkg -s linux-image-generic 2024-09-10T18:10:39.665 INFO:teuthology.orchestra.run.smithi151.stdout:Package: linux-image-generic 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Status: install ok installed 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Priority: optional 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Section: kernel 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Installed-Size: 20 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Maintainer: Ubuntu Kernel Team 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: amd64 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Source: linux-meta 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Version: 5.4.0.195.193 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Recommends: thermald 2024-09-10T18:10:39.666 INFO:teuthology.orchestra.run.smithi151.stdout:Description: Generic Linux kernel image 2024-09-10T18:10:39.667 INFO:teuthology.orchestra.run.smithi151.stdout: This package will always depend on the latest generic kernel image 2024-09-10T18:10:39.667 INFO:teuthology.orchestra.run.smithi151.stdout: available. 2024-09-10T18:10:39.667 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-10T18:10:39.667 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:10:39.667 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:10:39.667 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:10:39.667 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi151.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:10:39.668 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get clean 2024-09-10T18:10:39.755 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-09-10T18:10:39.820 INFO:teuthology.orchestra.run.smithi100.stdout:done 2024-09-10T18:10:39.907 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-10T18:10:40.006 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-10T18:10:40.088 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-10T18:10:40.169 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-10T18:10:40.220 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 20 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.4.0.195.193 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-10T18:10:40.296 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-09-10T18:10:40.297 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-09-10T18:10:40.297 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-09-10T18:10:40.297 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-09-10T18:10:40.297 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-10T18:10:40.297 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:10:40.297 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:10:40.297 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:10:40.297 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:10:40.298 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get clean 2024-09-10T18:10:40.379 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-09-10T18:10:40.637 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-10T18:10:40.637 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-10T18:10:40.707 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-10T18:10:40.789 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-10T18:10:41.758 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-09-10T18:10:41.785 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-10T18:10:41.854 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-09-10T18:10:42.073 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-09-10T18:10:42.073 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-09-10T18:10:42.313 INFO:teuthology.orchestra.run.smithi151.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-10T18:10:42.313 INFO:teuthology.orchestra.run.smithi151.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-10T18:10:42.315 DEBUG:teuthology.orchestra.run.smithi151:> dpkg -s linux-image-generic 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Package: linux-image-generic 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Status: install ok installed 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Priority: optional 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Section: kernel 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Installed-Size: 20 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Maintainer: Ubuntu Kernel Team 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: amd64 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Source: linux-meta 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Version: 5.4.0.195.193 2024-09-10T18:10:42.336 INFO:teuthology.orchestra.run.smithi151.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-10T18:10:42.337 INFO:teuthology.orchestra.run.smithi151.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-10T18:10:42.337 INFO:teuthology.orchestra.run.smithi151.stdout:Recommends: thermald 2024-09-10T18:10:42.337 INFO:teuthology.orchestra.run.smithi151.stdout:Description: Generic Linux kernel image 2024-09-10T18:10:42.337 INFO:teuthology.orchestra.run.smithi151.stdout: This package will always depend on the latest generic kernel image 2024-09-10T18:10:42.337 INFO:teuthology.orchestra.run.smithi151.stdout: available. 2024-09-10T18:10:42.337 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-09-10T18:10:42.342 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.GZQHzEBq87 2024-09-10T18:10:42.342 DEBUG:teuthology.orchestra.run.smithi151:> sudo cp /boot/grub/grub.cfg /tmp/tmp.GZQHzEBq87 2024-09-10T18:10:42.399 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.GZQHzEBq87 2024-09-10T18:10:42.447 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.GZQHzEBq87 is 9KB 2024-09-10T18:10:42.461 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.GZQHzEBq87 2024-09-10T18:10:42.467 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-10T18:10:42.483 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-10T18:10:42.509 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-10T18:10:42.522 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:10:42.523 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-10T18:10:42.523 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-10T18:10:42.560 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-10T18:10:42.561 DEBUG:teuthology.orchestra.run.smithi151:> sudo update-grub 2024-09-10T18:10:42.585 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-10T18:10:42.800 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-09-10T18:10:42.800 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-09-10T18:10:43.029 INFO:teuthology.orchestra.run.smithi100.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-10T18:10:43.030 INFO:teuthology.orchestra.run.smithi100.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-10T18:10:43.032 DEBUG:teuthology.orchestra.run.smithi100:> dpkg -s linux-image-generic 2024-09-10T18:10:43.053 INFO:teuthology.orchestra.run.smithi100.stdout:Package: linux-image-generic 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Status: install ok installed 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Priority: optional 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Section: kernel 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Installed-Size: 20 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Maintainer: Ubuntu Kernel Team 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: amd64 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Source: linux-meta 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Version: 5.4.0.195.193 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Recommends: thermald 2024-09-10T18:10:43.054 INFO:teuthology.orchestra.run.smithi100.stdout:Description: Generic Linux kernel image 2024-09-10T18:10:43.055 INFO:teuthology.orchestra.run.smithi100.stdout: This package will always depend on the latest generic kernel image 2024-09-10T18:10:43.055 INFO:teuthology.orchestra.run.smithi100.stdout: available. 2024-09-10T18:10:43.055 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-10T18:10:43.059 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.FlSVyYhxzs 2024-09-10T18:10:43.060 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub/grub.cfg /tmp/tmp.FlSVyYhxzs 2024-09-10T18:10:43.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.FlSVyYhxzs 2024-09-10T18:10:43.156 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.FlSVyYhxzs is 9KB 2024-09-10T18:10:43.171 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.FlSVyYhxzs 2024-09-10T18:10:43.177 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-10T18:10:43.232 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:10:43.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-10T18:10:43.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-10T18:10:43.273 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-10T18:10:43.273 DEBUG:teuthology.orchestra.run.smithi100:> sudo update-grub 2024-09-10T18:10:43.885 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub' 2024-09-10T18:10:43.896 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T18:10:43.906 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-09-10T18:10:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-10T18:10:44.160 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:44.557 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-10T18:10:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-10T18:10:44.657 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub' 2024-09-10T18:10:44.669 INFO:teuthology.orchestra.run.smithi100.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T18:10:44.681 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-09-10T18:10:44.930 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-10T18:10:44.942 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-10T18:10:45.295 INFO:teuthology.orchestra.run.smithi100.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-10T18:10:45.306 INFO:teuthology.orchestra.run.smithi100.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-10T18:10:45.573 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-09-10T18:10:45.575 DEBUG:teuthology.orchestra.run.smithi151:> sudo shutdown -r now 2024-09-10T18:10:46.386 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-09-10T18:10:46.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-09-10T18:11:15.606 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:11:15.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-10T18:11:15.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:11:16.391 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:11:16.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-10T18:11:16.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:11:50.205 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-09-10T18:11:50.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-10T18:11:59.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-10T18:11:59.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:11:59.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-10T18:11:59.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:12:17.597 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-09-10T18:12:18.365 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-09-10T18:12:29.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-10T18:12:29.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:12:29.997 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-09-10T18:12:30.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-10T18:12:30.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:12:30.789 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-10T18:12:30.951 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-10T18:12:30.952 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-10T18:12:30.952 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-09-10T18:12:30.999 INFO:teuthology.orchestra.run.smithi151.stdout:5.4.0-195-generic 2024-09-10T18:12:30.999 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-10T18:12:30.999 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:12:31.000 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T18:12:31.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-10T18:12:31.821 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-10T18:12:31.821 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-10T18:12:31.827 INFO:teuthology.orchestra.run.smithi100.stdout:5.4.0-195-generic 2024-09-10T18:12:31.827 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-10T18:12:31.828 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:12:31.828 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T18:12:32.001 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:12:32.001 DEBUG:teuthology.orchestra.run.smithi151:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:12:32.112 INFO:teuthology.orchestra.run.smithi151.stdout:ttyS1 2024-09-10T18:12:32.132 DEBUG:teuthology.parallel:result is None 2024-09-10T18:12:32.829 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:12:32.829 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:12:32.918 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-09-10T18:12:32.937 DEBUG:teuthology.parallel:result is None 2024-09-10T18:12:32.938 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:12:32.945 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:12:32.946 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:12:32.948 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:12:32.955 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:12:32.965 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:12:32.971 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:12:32.971 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:12:32.997 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:12:33.048 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:12:33.055 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:12:33.055 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:12:33.058 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:12:33.073 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:12:33.082 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:12:33.084 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:12:33.091 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:12:33.092 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:12:33.102 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:12:33.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:12:33.106 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:12:33.120 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:12:33.129 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:12:33.130 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:12:33.165 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:12:33.171 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:12:33.179 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:12:33.229 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:12:33.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:12:33.254 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:12:33.261 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:12:33.268 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:12:33.269 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:12:33.333 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-09-10T18:12:33.336 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-09-10T18:12:33.389 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:12:33.398 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:12:33.398 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:12:33.418 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:12:33.427 DEBUG:teuthology.task.selinux:Excluding smithi100: OS 'ubuntu' does not support SELinux 2024-09-10T18:12:33.427 DEBUG:teuthology.task.selinux:Excluding smithi151: OS 'ubuntu' does not support SELinux 2024-09-10T18:12:33.427 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:12:33.428 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-10T18:12:33.428 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:12:33.428 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:12:33.437 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:12:33.449 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:12:33.450 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:18:06.029 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-09-10T18:18:06.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-10T18:18:06.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:18:06.138 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-10T18:18:06.262 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-10T18:18:06.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-10T18:18:06.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:18:06.353 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-09-10T18:18:06.475 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-10T18:18:06.476 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:18:06.489 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:18:06.489 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:18:06.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:18:06.492 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:18:06.492 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:18:06.526 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-10T18:18:06.526 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Command line: ntpd -gq 2024-09-10T18:18:06.527 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-10T18:18:06.527 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Command line: ntpd -gq 2024-09-10T18:18:06.527 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: proto: precision = 0.078 usec (-24) 2024-09-10T18:18:06.527 INFO:teuthology.orchestra.run.smithi100.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T18:18:06.528 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T18:18:06.528 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T18:18:06.528 INFO:teuthology.orchestra.run.smithi100.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T18:18:06.529 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T18:18:06.529 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T18:18:06.529 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T18:18:06.529 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Listen normally on 3 enp3s0f1 172.21.15.100:123 2024-09-10T18:18:06.529 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Listen normally on 4 lo [::1]:123 2024-09-10T18:18:06.529 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d733%5]:123 2024-09-10T18:18:06.529 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:06 ntpd[13688]: Listening on routing socket on fd #22 for interface updates 2024-09-10T18:18:06.530 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: proto: precision = 0.182 usec (-22) 2024-09-10T18:18:06.531 INFO:teuthology.orchestra.run.smithi151.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T18:18:06.532 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T18:18:06.532 INFO:teuthology.orchestra.run.smithi151.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T18:18:06.533 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T18:18:06.533 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T18:18:06.533 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T18:18:06.533 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T18:18:06.534 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Listen normally on 3 enp3s0f1 172.21.15.151:123 2024-09-10T18:18:06.534 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Listen normally on 4 lo [::1]:123 2024-09-10T18:18:06.534 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f63%5]:123 2024-09-10T18:18:06.534 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:06 ntpd[13644]: Listening on routing socket on fd #22 for interface updates 2024-09-10T18:18:07.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:07.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:07.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:07.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:07.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:07 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:09 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:09 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:09 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:09 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:09 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:09 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:09.530 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:09.531 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:09 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:10.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:10.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:10 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:10.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:10 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:10.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:10 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:11 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:11 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:11 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:11 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:11 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:11.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:11 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:11.532 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:11 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:12.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:12.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:12.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:12 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:13.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:13 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:13.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:13 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:13.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:13 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:13.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:13 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:13.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:13 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:13.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:13 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:13.528 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:13.528 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:13.532 INFO:teuthology.orchestra.run.smithi151.stdout:10 Sep 18:18:13 ntpd[13644]: ntpd: time slew +0.005319 s 2024-09-10T18:18:13.532 INFO:teuthology.orchestra.run.smithi151.stdout:ntpd: time slew +0.005319s 2024-09-10T18:18:13.532 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T18:18:13.532 INFO:teuthology.orchestra.run.smithi151.stderr:10 Sep 18:18:13 ntpd[13644]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T18:18:13.595 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T18:18:13.595 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-09-10T18:18:13.595 INFO:teuthology.orchestra.run.smithi151.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:13.595 INFO:teuthology.orchestra.run.smithi151.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:13.595 INFO:teuthology.orchestra.run.smithi151.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:13.595 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:14.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:14.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:14.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:14.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:14.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:14.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:14.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:14.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:14.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:14.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:14.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:14.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:14 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stdout:10 Sep 18:18:15 ntpd[13688]: ntpd: time slew -0.000217 s 2024-09-10T18:18:15.527 INFO:teuthology.orchestra.run.smithi100.stdout:ntpd: time slew -0.000217s 2024-09-10T18:18:15.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T18:18:15.528 INFO:teuthology.orchestra.run.smithi100.stderr:10 Sep 18:18:15 ntpd[13688]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T18:18:15.602 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T18:18:15.603 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-09-10T18:18:15.603 INFO:teuthology.orchestra.run.smithi100.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:15.603 INFO:teuthology.orchestra.run.smithi100.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:15.603 INFO:teuthology.orchestra.run.smithi100.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:15.603 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-10T18:18:15.604 INFO:teuthology.run_tasks:Running task install... 2024-09-10T18:18:15.618 DEBUG:teuthology.task.install:project ceph 2024-09-10T18:18:15.618 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}} 2024-09-10T18:18:15.619 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:18:15.619 INFO:teuthology.task.install:Using flavor: default 2024-09-10T18:18:15.624 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T18:18:15.625 INFO:teuthology.task.install:extra packages: [] 2024-09-10T18:18:15.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-key list | grep Ceph 2024-09-10T18:18:15.626 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-key list | grep Ceph 2024-09-10T18:18:15.723 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-10T18:18:15.726 INFO:teuthology.orchestra.run.smithi100.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-10T18:18:15.783 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T18:18:15.783 INFO:teuthology.orchestra.run.smithi100.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T18:18:15.785 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-10T18:18:15.786 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:18:15.793 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T18:18:15.794 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T18:18:15.794 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-10T18:18:15.794 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:18:15.937 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default/ 2024-09-10T18:18:15.937 INFO:teuthology.task.install.deb:Package version is 17.2.7-1606-g5be57395-1focal 2024-09-10T18:18:15.940 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default/ 2024-09-10T18:18:15.941 INFO:teuthology.task.install.deb:Package version is 17.2.7-1606-g5be57395-1focal 2024-09-10T18:18:16.049 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:18:16.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T18:18:16.055 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:18:16.055 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T18:18:16.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo apt-get update 2024-09-10T18:18:16.071 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-09-10T18:18:16.212 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-10T18:18:16.217 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-10T18:18:16.235 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-10T18:18:16.258 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-10T18:18:16.263 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal InRelease 2024-09-10T18:18:16.264 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-10T18:18:16.282 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-10T18:18:16.284 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-10T18:18:16.307 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal Release.gpg 2024-09-10T18:18:16.312 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal InRelease 2024-09-10T18:18:16.328 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-10T18:18:16.328 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-10T18:18:16.333 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-10T18:18:16.358 INFO:teuthology.orchestra.run.smithi100.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal Release.gpg 2024-09-10T18:18:16.361 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main i386 Packages [3,376 B] 2024-09-10T18:18:16.376 INFO:teuthology.orchestra.run.smithi100.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-10T18:18:16.379 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main i386 Packages [3,376 B] 2024-09-10T18:18:16.411 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-10T18:18:16.861 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 28.4 kB in 1s (45.2 kB/s) 2024-09-10T18:18:16.916 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 28.4 kB in 1s (44.4 kB/s) 2024-09-10T18:18:18.007 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-09-10T18:18:18.035 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1606-g5be57395-1focal cephadm=17.2.7-1606-g5be57395-1focal ceph-mds=17.2.7-1606-g5be57395-1focal ceph-mgr=17.2.7-1606-g5be57395-1focal ceph-common=17.2.7-1606-g5be57395-1focal ceph-fuse=17.2.7-1606-g5be57395-1focal ceph-test=17.2.7-1606-g5be57395-1focal ceph-volume=17.2.7-1606-g5be57395-1focal radosgw=17.2.7-1606-g5be57395-1focal python3-rados=17.2.7-1606-g5be57395-1focal python3-rgw=17.2.7-1606-g5be57395-1focal python3-cephfs=17.2.7-1606-g5be57395-1focal python3-rbd=17.2.7-1606-g5be57395-1focal libcephfs2=17.2.7-1606-g5be57395-1focal libcephfs-dev=17.2.7-1606-g5be57395-1focal librados2=17.2.7-1606-g5be57395-1focal librbd1=17.2.7-1606-g5be57395-1focal rbd-fuse=17.2.7-1606-g5be57395-1focal 2024-09-10T18:18:18.072 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-10T18:18:18.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1606-g5be57395-1focal cephadm=17.2.7-1606-g5be57395-1focal ceph-mds=17.2.7-1606-g5be57395-1focal ceph-mgr=17.2.7-1606-g5be57395-1focal ceph-common=17.2.7-1606-g5be57395-1focal ceph-fuse=17.2.7-1606-g5be57395-1focal ceph-test=17.2.7-1606-g5be57395-1focal ceph-volume=17.2.7-1606-g5be57395-1focal radosgw=17.2.7-1606-g5be57395-1focal python3-rados=17.2.7-1606-g5be57395-1focal python3-rgw=17.2.7-1606-g5be57395-1focal python3-cephfs=17.2.7-1606-g5be57395-1focal python3-rbd=17.2.7-1606-g5be57395-1focal libcephfs2=17.2.7-1606-g5be57395-1focal libcephfs-dev=17.2.7-1606-g5be57395-1focal librados2=17.2.7-1606-g5be57395-1focal librbd1=17.2.7-1606-g5be57395-1focal rbd-fuse=17.2.7-1606-g5be57395-1focal 2024-09-10T18:18:18.105 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-09-10T18:18:18.172 INFO:teuthology.orchestra.run.smithi100.stdout:Reading package lists... 2024-09-10T18:18:18.323 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-09-10T18:18:18.323 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-09-10T18:18:18.391 INFO:teuthology.orchestra.run.smithi100.stdout:Building dependency tree... 2024-09-10T18:18:18.391 INFO:teuthology.orchestra.run.smithi100.stdout:Reading state information... 2024-09-10T18:18:18.528 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-10T18:18:18.530 INFO:teuthology.orchestra.run.smithi151.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-10T18:18:18.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-10T18:18:18.533 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T18:18:18.533 INFO:teuthology.orchestra.run.smithi151.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-10T18:18:18.533 INFO:teuthology.orchestra.run.smithi151.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-10T18:18:18.533 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-10T18:18:18.533 INFO:teuthology.orchestra.run.smithi151.stdout: socat xmlstarlet 2024-09-10T18:18:18.533 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-09-10T18:18:18.533 INFO:teuthology.orchestra.run.smithi151.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout:Recommended packages: 2024-09-10T18:18:18.534 INFO:teuthology.orchestra.run.smithi151.stdout: btrfs-tools 2024-09-10T18:18:18.588 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T18:18:18.589 INFO:teuthology.orchestra.run.smithi100.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-10T18:18:18.590 INFO:teuthology.orchestra.run.smithi100.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T18:18:18.590 INFO:teuthology.orchestra.run.smithi100.stdout:The following additional packages will be installed: 2024-09-10T18:18:18.590 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T18:18:18.590 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-10T18:18:18.590 INFO:teuthology.orchestra.run.smithi100.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-10T18:18:18.590 INFO:teuthology.orchestra.run.smithi100.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-10T18:18:18.590 INFO:teuthology.orchestra.run.smithi100.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-10T18:18:18.591 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: socat xmlstarlet 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout:Suggested packages: 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-10T18:18:18.592 INFO:teuthology.orchestra.run.smithi100.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout:Recommended packages: 2024-09-10T18:18:18.593 INFO:teuthology.orchestra.run.smithi100.stdout: btrfs-tools 2024-09-10T18:18:18.654 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-09-10T18:18:18.654 INFO:teuthology.orchestra.run.smithi151.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T18:18:18.654 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T18:18:18.654 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-10T18:18:18.656 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-10T18:18:18.657 INFO:teuthology.orchestra.run.smithi151.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-10T18:18:18.661 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages will be upgraded: 2024-09-10T18:18:18.661 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 librbd1 2024-09-10T18:18:18.713 INFO:teuthology.orchestra.run.smithi100.stdout:The following NEW packages will be installed: 2024-09-10T18:18:18.713 INFO:teuthology.orchestra.run.smithi100.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T18:18:18.714 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T18:18:18.714 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-10T18:18:18.715 INFO:teuthology.orchestra.run.smithi100.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-10T18:18:18.716 INFO:teuthology.orchestra.run.smithi100.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-10T18:18:18.717 INFO:teuthology.orchestra.run.smithi100.stdout:The following packages will be upgraded: 2024-09-10T18:18:18.717 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 librbd1 2024-09-10T18:18:18.722 INFO:teuthology.orchestra.run.smithi151.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-10T18:18:18.722 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 201 MB of archives. 2024-09-10T18:18:18.722 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-10T18:18:18.722 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-10T18:18:18.806 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-10T18:18:18.808 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-10T18:18:18.813 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1606-g5be57395-1focal [3,507 kB] 2024-09-10T18:18:18.817 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-10T18:18:18.823 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-10T18:18:18.848 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-10T18:18:18.848 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-10T18:18:18.854 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-10T18:18:18.865 INFO:teuthology.orchestra.run.smithi100.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-10T18:18:18.865 INFO:teuthology.orchestra.run.smithi100.stdout:Need to get 201 MB of archives. 2024-09-10T18:18:18.865 INFO:teuthology.orchestra.run.smithi100.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-10T18:18:18.865 INFO:teuthology.orchestra.run.smithi100.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1606-g5be57395-1focal [3,507 kB] 2024-09-10T18:18:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-10T18:18:18.888 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-10T18:18:18.894 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-10T18:18:18.896 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-10T18:18:18.896 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-10T18:18:18.898 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-10T18:18:18.899 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-10T18:18:18.899 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-10T18:18:18.899 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-10T18:18:18.904 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-10T18:18:18.909 INFO:teuthology.orchestra.run.smithi100.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-10T18:18:18.919 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-10T18:18:18.919 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-10T18:18:18.920 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-10T18:18:18.921 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-10T18:18:18.922 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T18:18:18.923 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-10T18:18:18.923 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-10T18:18:18.925 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-10T18:18:18.926 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-10T18:18:18.939 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-10T18:18:18.940 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-10T18:18:18.941 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-10T18:18:18.957 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-10T18:18:18.957 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-10T18:18:18.958 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-10T18:18:18.959 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-10T18:18:18.960 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-10T18:18:18.963 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-10T18:18:18.965 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T18:18:18.974 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-10T18:18:18.975 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-10T18:18:18.976 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-10T18:18:18.989 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1606-g5be57395-1focal [3,560 kB] 2024-09-10T18:18:18.992 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-10T18:18:18.992 INFO:teuthology.orchestra.run.smithi151.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-10T18:18:18.993 INFO:teuthology.orchestra.run.smithi151.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-10T18:18:19.048 INFO:teuthology.orchestra.run.smithi151.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1606-g5be57395-1focal [761 kB] 2024-09-10T18:18:19.073 INFO:teuthology.orchestra.run.smithi151.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-10T18:18:19.088 INFO:teuthology.orchestra.run.smithi151.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-10T18:18:19.096 INFO:teuthology.orchestra.run.smithi151.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1606-g5be57395-1focal [414 kB] 2024-09-10T18:18:19.140 INFO:teuthology.orchestra.run.smithi100.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1606-g5be57395-1focal [3,560 kB] 2024-09-10T18:18:19.155 INFO:teuthology.orchestra.run.smithi151.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-10T18:18:19.161 INFO:teuthology.orchestra.run.smithi151.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1606-g5be57395-1focal [22.6 kB] 2024-09-10T18:18:19.163 INFO:teuthology.orchestra.run.smithi151.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1606-g5be57395-1focal [200 kB] 2024-09-10T18:18:19.167 INFO:teuthology.orchestra.run.smithi151.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1606-g5be57395-1focal [49.1 kB] 2024-09-10T18:18:19.169 INFO:teuthology.orchestra.run.smithi151.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1606-g5be57395-1focal [389 kB] 2024-09-10T18:18:19.174 INFO:teuthology.orchestra.run.smithi151.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-10T18:18:19.174 INFO:teuthology.orchestra.run.smithi151.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-10T18:18:19.175 INFO:teuthology.orchestra.run.smithi151.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-10T18:18:19.175 INFO:teuthology.orchestra.run.smithi151.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-10T18:18:19.176 INFO:teuthology.orchestra.run.smithi151.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-10T18:18:19.176 INFO:teuthology.orchestra.run.smithi151.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-10T18:18:19.177 INFO:teuthology.orchestra.run.smithi151.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-10T18:18:19.177 INFO:teuthology.orchestra.run.smithi151.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-10T18:18:19.177 INFO:teuthology.orchestra.run.smithi151.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-10T18:18:19.192 INFO:teuthology.orchestra.run.smithi151.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-10T18:18:19.194 INFO:teuthology.orchestra.run.smithi151.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1606-g5be57395-1focal [5,637 kB] 2024-09-10T18:18:19.194 INFO:teuthology.orchestra.run.smithi151.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-10T18:18:19.209 INFO:teuthology.orchestra.run.smithi151.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-10T18:18:19.212 INFO:teuthology.orchestra.run.smithi151.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-10T18:18:19.212 INFO:teuthology.orchestra.run.smithi151.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-10T18:18:19.217 INFO:teuthology.orchestra.run.smithi151.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-10T18:18:19.219 INFO:teuthology.orchestra.run.smithi151.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-10T18:18:19.235 INFO:teuthology.orchestra.run.smithi151.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-09-10T18:18:19.238 INFO:teuthology.orchestra.run.smithi151.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-10T18:18:19.253 INFO:teuthology.orchestra.run.smithi100.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1606-g5be57395-1focal [761 kB] 2024-09-10T18:18:19.273 INFO:teuthology.orchestra.run.smithi100.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1606-g5be57395-1focal [414 kB] 2024-09-10T18:18:19.279 INFO:teuthology.orchestra.run.smithi151.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-10T18:18:19.280 INFO:teuthology.orchestra.run.smithi151.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-10T18:18:19.280 INFO:teuthology.orchestra.run.smithi151.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-10T18:18:19.280 INFO:teuthology.orchestra.run.smithi151.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-10T18:18:19.281 INFO:teuthology.orchestra.run.smithi151.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-10T18:18:19.281 INFO:teuthology.orchestra.run.smithi151.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-10T18:18:19.284 INFO:teuthology.orchestra.run.smithi100.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1606-g5be57395-1focal [22.6 kB] 2024-09-10T18:18:19.285 INFO:teuthology.orchestra.run.smithi100.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1606-g5be57395-1focal [200 kB] 2024-09-10T18:18:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1606-g5be57395-1focal [49.1 kB] 2024-09-10T18:18:19.289 INFO:teuthology.orchestra.run.smithi100.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1606-g5be57395-1focal [389 kB] 2024-09-10T18:18:19.293 INFO:teuthology.orchestra.run.smithi151.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T18:18:19.294 INFO:teuthology.orchestra.run.smithi151.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-10T18:18:19.294 INFO:teuthology.orchestra.run.smithi151.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-10T18:18:19.297 INFO:teuthology.orchestra.run.smithi151.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-10T18:18:19.298 INFO:teuthology.orchestra.run.smithi100.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1606-g5be57395-1focal [5,637 kB] 2024-09-10T18:18:19.312 INFO:teuthology.orchestra.run.smithi100.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-10T18:18:19.314 INFO:teuthology.orchestra.run.smithi151.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-10T18:18:19.315 INFO:teuthology.orchestra.run.smithi151.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-10T18:18:19.315 INFO:teuthology.orchestra.run.smithi151.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-10T18:18:19.317 INFO:teuthology.orchestra.run.smithi151.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-10T18:18:19.318 INFO:teuthology.orchestra.run.smithi151.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-10T18:18:19.320 INFO:teuthology.orchestra.run.smithi100.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-10T18:18:19.363 INFO:teuthology.orchestra.run.smithi100.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-10T18:18:19.372 INFO:teuthology.orchestra.run.smithi151.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1606-g5be57395-1focal [116 kB] 2024-09-10T18:18:19.373 INFO:teuthology.orchestra.run.smithi151.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1606-g5be57395-1focal [479 kB] 2024-09-10T18:18:19.376 INFO:teuthology.orchestra.run.smithi100.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-10T18:18:19.382 INFO:teuthology.orchestra.run.smithi151.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1606-g5be57395-1focal [27.0 MB] 2024-09-10T18:18:19.502 INFO:teuthology.orchestra.run.smithi100.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-10T18:18:19.508 INFO:teuthology.orchestra.run.smithi100.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-10T18:18:19.537 INFO:teuthology.orchestra.run.smithi100.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-10T18:18:19.566 INFO:teuthology.orchestra.run.smithi100.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1606-g5be57395-1focal [116 kB] 2024-09-10T18:18:19.573 INFO:teuthology.orchestra.run.smithi100.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1606-g5be57395-1focal [479 kB] 2024-09-10T18:18:19.588 INFO:teuthology.orchestra.run.smithi100.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1606-g5be57395-1focal [27.0 MB] 2024-09-10T18:18:19.679 INFO:teuthology.orchestra.run.smithi100.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-10T18:18:19.686 INFO:teuthology.orchestra.run.smithi100.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-10T18:18:19.709 INFO:teuthology.orchestra.run.smithi100.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-10T18:18:19.716 INFO:teuthology.orchestra.run.smithi100.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-10T18:18:19.717 INFO:teuthology.orchestra.run.smithi100.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-10T18:18:19.723 INFO:teuthology.orchestra.run.smithi100.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-10T18:18:19.724 INFO:teuthology.orchestra.run.smithi100.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-10T18:18:19.725 INFO:teuthology.orchestra.run.smithi100.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-10T18:18:19.726 INFO:teuthology.orchestra.run.smithi100.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-10T18:18:19.761 INFO:teuthology.orchestra.run.smithi100.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-10T18:18:19.813 INFO:teuthology.orchestra.run.smithi100.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-10T18:18:19.814 INFO:teuthology.orchestra.run.smithi100.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-10T18:18:19.815 INFO:teuthology.orchestra.run.smithi100.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-10T18:18:19.816 INFO:teuthology.orchestra.run.smithi100.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-10T18:18:19.818 INFO:teuthology.orchestra.run.smithi100.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T18:18:19.819 INFO:teuthology.orchestra.run.smithi100.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-10T18:18:19.819 INFO:teuthology.orchestra.run.smithi100.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-10T18:18:19.822 INFO:teuthology.orchestra.run.smithi100.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-10T18:18:19.842 INFO:teuthology.orchestra.run.smithi100.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-10T18:18:19.905 INFO:teuthology.orchestra.run.smithi100.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-10T18:18:19.907 INFO:teuthology.orchestra.run.smithi100.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-10T18:18:19.908 INFO:teuthology.orchestra.run.smithi100.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-10T18:18:19.917 INFO:teuthology.orchestra.run.smithi100.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-10T18:18:19.918 INFO:teuthology.orchestra.run.smithi100.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-10T18:18:19.919 INFO:teuthology.orchestra.run.smithi100.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-10T18:18:19.919 INFO:teuthology.orchestra.run.smithi100.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-10T18:18:19.925 INFO:teuthology.orchestra.run.smithi100.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-10T18:18:19.927 INFO:teuthology.orchestra.run.smithi100.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-10T18:18:20.005 INFO:teuthology.orchestra.run.smithi100.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T18:18:20.006 INFO:teuthology.orchestra.run.smithi100.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-10T18:18:20.009 INFO:teuthology.orchestra.run.smithi100.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-10T18:18:20.011 INFO:teuthology.orchestra.run.smithi100.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-10T18:18:20.015 INFO:teuthology.orchestra.run.smithi100.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-10T18:18:20.016 INFO:teuthology.orchestra.run.smithi100.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-10T18:18:20.017 INFO:teuthology.orchestra.run.smithi100.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-10T18:18:20.297 INFO:teuthology.orchestra.run.smithi151.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1606-g5be57395-1focal [5,178 kB] 2024-09-10T18:18:20.389 INFO:teuthology.orchestra.run.smithi100.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-10T18:18:20.402 INFO:teuthology.orchestra.run.smithi151.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1606-g5be57395-1focal [223 kB] 2024-09-10T18:18:20.410 INFO:teuthology.orchestra.run.smithi100.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-10T18:18:20.412 INFO:teuthology.orchestra.run.smithi100.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-10T18:18:20.418 INFO:teuthology.orchestra.run.smithi151.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1606-g5be57395-1focal [157 kB] 2024-09-10T18:18:20.421 INFO:teuthology.orchestra.run.smithi151.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1606-g5be57395-1focal [1,593 kB] 2024-09-10T18:18:20.496 INFO:teuthology.orchestra.run.smithi151.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1606-g5be57395-1focal [4,686 kB] 2024-09-10T18:18:20.522 INFO:teuthology.orchestra.run.smithi100.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-10T18:18:20.522 INFO:teuthology.orchestra.run.smithi100.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-10T18:18:20.599 INFO:teuthology.orchestra.run.smithi100.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-10T18:18:20.600 INFO:teuthology.orchestra.run.smithi100.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1606-g5be57395-1focal [5,178 kB] 2024-09-10T18:18:20.600 INFO:teuthology.orchestra.run.smithi100.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-10T18:18:20.601 INFO:teuthology.orchestra.run.smithi100.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-10T18:18:20.602 INFO:teuthology.orchestra.run.smithi100.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-10T18:18:20.602 INFO:teuthology.orchestra.run.smithi100.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-10T18:18:20.604 INFO:teuthology.orchestra.run.smithi100.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-10T18:18:20.605 INFO:teuthology.orchestra.run.smithi100.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-10T18:18:20.608 INFO:teuthology.orchestra.run.smithi151.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1606-g5be57395-1focal [23.1 MB] 2024-09-10T18:18:20.635 INFO:teuthology.orchestra.run.smithi100.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-10T18:18:20.638 INFO:teuthology.orchestra.run.smithi100.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-10T18:18:20.657 INFO:teuthology.orchestra.run.smithi100.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1606-g5be57395-1focal [223 kB] 2024-09-10T18:18:20.660 INFO:teuthology.orchestra.run.smithi100.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1606-g5be57395-1focal [157 kB] 2024-09-10T18:18:20.663 INFO:teuthology.orchestra.run.smithi100.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1606-g5be57395-1focal [1,593 kB] 2024-09-10T18:18:20.696 INFO:teuthology.orchestra.run.smithi100.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1606-g5be57395-1focal [4,686 kB] 2024-09-10T18:18:20.716 INFO:teuthology.orchestra.run.smithi100.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-10T18:18:20.719 INFO:teuthology.orchestra.run.smithi100.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-10T18:18:20.721 INFO:teuthology.orchestra.run.smithi100.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-10T18:18:20.727 INFO:teuthology.orchestra.run.smithi100.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-10T18:18:20.731 INFO:teuthology.orchestra.run.smithi100.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-10T18:18:20.800 INFO:teuthology.orchestra.run.smithi100.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-09-10T18:18:20.805 INFO:teuthology.orchestra.run.smithi100.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-10T18:18:20.813 INFO:teuthology.orchestra.run.smithi100.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1606-g5be57395-1focal [23.1 MB] 2024-09-10T18:18:20.972 INFO:teuthology.orchestra.run.smithi100.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-10T18:18:20.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-10T18:18:20.976 INFO:teuthology.orchestra.run.smithi100.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-10T18:18:20.977 INFO:teuthology.orchestra.run.smithi100.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-10T18:18:20.977 INFO:teuthology.orchestra.run.smithi100.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-10T18:18:20.978 INFO:teuthology.orchestra.run.smithi100.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-10T18:18:20.992 INFO:teuthology.orchestra.run.smithi100.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T18:18:20.993 INFO:teuthology.orchestra.run.smithi100.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-10T18:18:20.993 INFO:teuthology.orchestra.run.smithi100.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-10T18:18:21.054 INFO:teuthology.orchestra.run.smithi100.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-10T18:18:21.136 INFO:teuthology.orchestra.run.smithi100.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-10T18:18:21.137 INFO:teuthology.orchestra.run.smithi100.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-10T18:18:21.137 INFO:teuthology.orchestra.run.smithi100.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-10T18:18:21.140 INFO:teuthology.orchestra.run.smithi100.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-10T18:18:21.141 INFO:teuthology.orchestra.run.smithi100.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-10T18:18:21.445 INFO:teuthology.orchestra.run.smithi100.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1606-g5be57395-1focal [3,936 B] 2024-09-10T18:18:21.446 INFO:teuthology.orchestra.run.smithi100.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1606-g5be57395-1focal [871 kB] 2024-09-10T18:18:21.456 INFO:teuthology.orchestra.run.smithi100.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1606-g5be57395-1focal [2,415 kB] 2024-09-10T18:18:21.474 INFO:teuthology.orchestra.run.smithi100.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1606-g5be57395-1focal [81.4 kB] 2024-09-10T18:18:21.475 INFO:teuthology.orchestra.run.smithi100.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1606-g5be57395-1focal [120 kB] 2024-09-10T18:18:21.476 INFO:teuthology.orchestra.run.smithi100.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1606-g5be57395-1focal [6,653 kB] 2024-09-10T18:18:21.645 INFO:teuthology.orchestra.run.smithi100.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1606-g5be57395-1focal [7,889 kB] 2024-09-10T18:18:21.795 INFO:teuthology.orchestra.run.smithi151.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1606-g5be57395-1focal [3,936 B] 2024-09-10T18:18:21.795 INFO:teuthology.orchestra.run.smithi151.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1606-g5be57395-1focal [871 kB] 2024-09-10T18:18:21.809 INFO:teuthology.orchestra.run.smithi151.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1606-g5be57395-1focal [2,415 kB] 2024-09-10T18:18:21.833 INFO:teuthology.orchestra.run.smithi100.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1606-g5be57395-1focal [4,052 B] 2024-09-10T18:18:21.833 INFO:teuthology.orchestra.run.smithi100.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1606-g5be57395-1focal [53.4 MB] 2024-09-10T18:18:21.889 INFO:teuthology.orchestra.run.smithi151.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1606-g5be57395-1focal [81.4 kB] 2024-09-10T18:18:21.890 INFO:teuthology.orchestra.run.smithi151.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1606-g5be57395-1focal [120 kB] 2024-09-10T18:18:21.893 INFO:teuthology.orchestra.run.smithi151.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1606-g5be57395-1focal [6,653 kB] 2024-09-10T18:18:22.210 INFO:teuthology.orchestra.run.smithi151.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1606-g5be57395-1focal [7,889 kB] 2024-09-10T18:18:22.513 INFO:teuthology.orchestra.run.smithi151.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1606-g5be57395-1focal [4,052 B] 2024-09-10T18:18:22.513 INFO:teuthology.orchestra.run.smithi151.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1606-g5be57395-1focal [53.4 MB] 2024-09-10T18:18:23.605 INFO:teuthology.orchestra.run.smithi100.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1606-g5be57395-1focal [231 kB] 2024-09-10T18:18:23.610 INFO:teuthology.orchestra.run.smithi100.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1606-g5be57395-1focal [21.9 kB] 2024-09-10T18:18:23.611 INFO:teuthology.orchestra.run.smithi100.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1606-g5be57395-1focal [17.2 MB] 2024-09-10T18:18:24.133 INFO:teuthology.orchestra.run.smithi100.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1606-g5be57395-1focal [80.7 kB] 2024-09-10T18:18:24.404 INFO:teuthology.orchestra.run.smithi151.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1606-g5be57395-1focal [231 kB] 2024-09-10T18:18:24.409 INFO:teuthology.orchestra.run.smithi151.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1606-g5be57395-1focal [21.9 kB] 2024-09-10T18:18:24.410 INFO:teuthology.orchestra.run.smithi151.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1606-g5be57395-1focal [17.2 MB] 2024-09-10T18:18:24.725 INFO:teuthology.orchestra.run.smithi151.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1606-g5be57395-1focal [80.7 kB] 2024-09-10T18:18:24.778 INFO:teuthology.orchestra.run.smithi100.stdout:Fetched 201 MB in 5s (37.2 MB/s) 2024-09-10T18:18:24.900 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-10T18:18:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-10T18:18:24.962 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-10T18:18:25.031 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-10T18:18:25.381 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 201 MB in 6s (33.2 MB/s) 2024-09-10T18:18:25.395 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../001-librbd1_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:25.515 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-10T18:18:25.521 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librbd1 (17.2.7-1606-g5be57395-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-10T18:18:25.572 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-10T18:18:25.577 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-10T18:18:25.655 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-10T18:18:25.996 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../001-librbd1_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:26.118 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librbd1 (17.2.7-1606-g5be57395-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-10T18:18:26.241 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../002-librados2_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:26.367 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librados2 (17.2.7-1606-g5be57395-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-10T18:18:26.847 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../002-librados2_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:26.957 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librados2 (17.2.7-1606-g5be57395-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-10T18:18:27.030 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T18:18:27.147 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:27.147 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:27.415 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rados. 2024-09-10T18:18:27.433 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:27.473 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rados (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:27.637 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T18:18:27.654 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:27.695 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:27.826 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T18:18:27.842 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:27.884 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-argparse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:28.056 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rados. 2024-09-10T18:18:28.073 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:28.114 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rados (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:28.154 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T18:18:28.171 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:28.212 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cephfs (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:28.468 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T18:18:28.484 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:28.518 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-argparse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:28.539 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T18:18:28.557 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:28.598 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:28.737 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T18:18:28.756 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:28.796 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cephfs (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:28.893 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T18:18:28.912 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-10T18:18:28.943 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-10T18:18:29.124 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T18:18:29.144 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:29.191 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:29.220 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T18:18:29.237 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:29.279 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rbd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:29.536 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T18:18:29.556 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-10T18:18:29.595 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-10T18:18:29.631 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-10T18:18:29.647 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-10T18:18:29.689 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-10T18:18:29.831 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T18:18:29.850 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:29.881 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rbd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:30.059 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-10T18:18:30.079 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-10T18:18:30.117 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-10T18:18:30.260 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-10T18:18:30.277 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-10T18:18:30.318 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-10T18:18:30.429 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T18:18:30.447 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-10T18:18:30.487 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-10T18:18:30.680 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-10T18:18:30.697 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-10T18:18:30.730 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-10T18:18:30.882 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T18:18:30.897 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-10T18:18:30.940 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-10T18:18:31.042 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T18:18:31.061 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-10T18:18:31.100 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-10T18:18:31.276 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T18:18:31.293 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-10T18:18:31.334 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-10T18:18:31.479 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T18:18:31.495 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-10T18:18:31.537 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-10T18:18:31.805 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T18:18:31.821 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-10T18:18:31.824 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T18:18:31.839 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-10T18:18:31.873 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:18:31.873 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-10T18:18:32.353 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T18:18:32.372 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-10T18:18:32.374 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T18:18:32.390 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-10T18:18:32.422 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:18:32.432 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:18:32.803 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T18:18:32.820 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-10T18:18:32.861 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:18:32.923 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T18:18:32.941 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-10T18:18:32.973 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:18:33.239 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-10T18:18:33.255 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-10T18:18:33.297 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-10T18:18:33.352 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T18:18:33.369 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-10T18:18:33.410 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:18:33.658 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package librgw2. 2024-09-10T18:18:33.674 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../019-librgw2_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:33.716 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking librgw2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:33.764 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-10T18:18:33.780 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-10T18:18:33.814 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-10T18:18:34.176 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librgw2. 2024-09-10T18:18:34.196 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../019-librgw2_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:34.234 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librgw2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:34.703 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T18:18:34.720 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:34.761 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:35.013 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T18:18:35.030 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:35.071 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:35.088 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liboath0. 2024-09-10T18:18:35.105 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-10T18:18:35.147 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-10T18:18:35.450 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liboath0. 2024-09-10T18:18:35.466 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-10T18:18:35.483 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T18:18:35.503 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:35.533 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-10T18:18:35.541 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libradosstriper1 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:35.820 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T18:18:35.835 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:35.870 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libradosstriper1 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:35.986 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-common. 2024-09-10T18:18:36.002 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:36.044 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:36.340 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-common. 2024-09-10T18:18:36.357 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:36.398 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:38.416 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-base. 2024-09-10T18:18:38.435 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:38.485 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-base (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:39.097 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-base. 2024-09-10T18:18:39.114 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:39.166 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-base (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:39.304 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cryptography. 2024-09-10T18:18:39.323 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-10T18:18:39.354 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-10T18:18:39.683 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T18:18:39.700 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-10T18:18:39.732 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-10T18:18:40.044 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-packaging. 2024-09-10T18:18:40.059 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-10T18:18:40.087 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cryptography. 2024-09-10T18:18:40.106 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-10T18:18:40.144 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-10T18:18:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-10T18:18:40.396 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T18:18:40.412 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:40.454 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:40.524 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T18:18:40.544 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-10T18:18:40.582 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-10T18:18:40.843 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T18:18:40.858 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-10T18:18:40.876 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-packaging. 2024-09-10T18:18:40.896 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-10T18:18:40.900 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-10T18:18:40.968 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-10T18:18:41.204 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T18:18:41.221 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T18:18:41.223 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-10T18:18:41.237 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:41.262 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-10T18:18:41.279 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:41.590 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T18:18:41.608 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-10T18:18:41.640 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-10T18:18:41.675 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T18:18:41.693 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-10T18:18:41.733 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-10T18:18:41.977 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jwt. 2024-09-10T18:18:41.996 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-10T18:18:42.035 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-10T18:18:42.061 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T18:18:42.078 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-10T18:18:42.119 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-10T18:18:42.347 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T18:18:42.362 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-10T18:18:42.404 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-10T18:18:42.456 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T18:18:42.472 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-10T18:18:42.514 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-10T18:18:42.708 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T18:18:42.725 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-10T18:18:42.765 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-10T18:18:42.850 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jwt. 2024-09-10T18:18:42.866 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-10T18:18:42.908 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-10T18:18:43.068 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-mako. 2024-09-10T18:18:43.085 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-10T18:18:43.126 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-10T18:18:43.311 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T18:18:43.327 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-10T18:18:43.370 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-10T18:18:43.438 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T18:18:43.457 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T18:18:43.496 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T18:18:43.689 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T18:18:43.706 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-10T18:18:43.747 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-10T18:18:43.818 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T18:18:43.836 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-10T18:18:43.882 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-10T18:18:44.058 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-mako. 2024-09-10T18:18:44.075 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-10T18:18:44.116 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-10T18:18:44.185 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webob. 2024-09-10T18:18:44.202 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-10T18:18:44.243 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-10T18:18:44.453 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T18:18:44.472 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T18:18:44.511 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T18:18:44.572 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-10T18:18:44.589 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-10T18:18:44.630 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-10T18:18:44.822 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T18:18:44.841 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-10T18:18:44.880 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-10T18:18:44.932 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-bs4. 2024-09-10T18:18:44.949 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-10T18:18:44.991 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-10T18:18:45.183 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webob. 2024-09-10T18:18:45.199 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-10T18:18:45.241 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-10T18:18:45.327 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T18:18:45.343 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-10T18:18:45.388 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-10T18:18:45.619 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-10T18:18:45.634 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-10T18:18:45.685 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-10T18:18:45.730 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T18:18:45.748 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-10T18:18:45.794 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-10T18:18:46.021 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-bs4. 2024-09-10T18:18:46.038 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-10T18:18:46.080 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-10T18:18:46.106 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-paste. 2024-09-10T18:18:46.122 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-10T18:18:46.164 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-10T18:18:46.458 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T18:18:46.476 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T18:18:46.477 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-10T18:18:46.492 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-10T18:18:46.519 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-10T18:18:46.534 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-10T18:18:46.828 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T18:18:46.847 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T18:18:46.848 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-10T18:18:46.866 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-10T18:18:46.887 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-10T18:18:46.906 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-10T18:18:47.190 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T18:18:47.206 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-10T18:18:47.243 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-paste. 2024-09-10T18:18:47.248 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-10T18:18:47.259 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-10T18:18:47.301 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-10T18:18:47.609 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T18:18:47.625 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-10T18:18:47.667 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-10T18:18:47.670 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T18:18:47.687 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-10T18:18:47.729 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-10T18:18:48.014 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T18:18:48.031 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T18:18:48.032 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-10T18:18:48.050 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-10T18:18:48.072 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-10T18:18:48.114 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-10T18:18:48.392 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T18:18:48.401 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T18:18:48.412 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:48.420 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-10T18:18:48.459 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-10T18:18:48.475 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:48.803 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T18:18:48.821 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-10T18:18:48.824 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T18:18:48.841 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:48.862 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-10T18:18:48.882 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:49.250 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T18:18:49.267 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-10T18:18:49.308 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-10T18:18:49.386 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T18:18:49.404 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:49.452 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mon (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:49.644 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T18:18:49.660 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:49.727 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:50.935 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T18:18:50.953 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:50.992 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:51.047 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T18:18:51.065 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:51.096 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-osd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:51.529 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T18:18:51.545 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:51.587 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mon (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:52.406 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T18:18:52.425 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:52.464 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-osd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:53.283 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph. 2024-09-10T18:18:53.301 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../053-ceph_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:53.341 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:53.660 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T18:18:53.677 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:53.718 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:54.131 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T18:18:54.151 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:54.180 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mds (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:54.609 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph. 2024-09-10T18:18:54.628 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../053-ceph_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:54.692 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package cephadm. 2024-09-10T18:18:54.700 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:54.708 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../056-cephadm_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:54.750 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:55.011 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T18:18:55.029 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:55.069 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:55.078 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T18:18:55.096 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-10T18:18:55.136 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-10T18:18:55.447 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T18:18:55.467 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:55.489 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T18:18:55.506 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:55.506 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:55.547 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mds (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:55.842 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T18:18:55.858 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T18:18:55.900 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T18:18:56.117 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephadm. 2024-09-10T18:18:56.137 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../056-cephadm_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:18:56.175 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:56.228 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjs-underscore. 2024-09-10T18:18:56.245 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-10T18:18:56.286 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-10T18:18:56.520 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T18:18:56.538 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-10T18:18:56.556 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-10T18:18:56.571 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-10T18:18:56.577 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-10T18:18:56.613 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-10T18:18:56.906 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T18:18:56.925 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:56.964 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:56.991 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-routes. 2024-09-10T18:18:57.007 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-10T18:18:57.049 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-10T18:18:57.283 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T18:18:57.301 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T18:18:57.341 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T18:18:57.361 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T18:18:57.380 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:57.418 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:57.577 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjs-underscore. 2024-09-10T18:18:57.594 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-10T18:18:57.627 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-10T18:18:57.897 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-10T18:18:57.916 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-10T18:18:57.980 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-10T18:18:58.299 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-routes. 2024-09-10T18:18:58.315 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-10T18:18:58.357 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-10T18:18:58.701 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T18:18:58.718 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:18:58.734 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-decorator. 2024-09-10T18:18:58.753 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-10T18:18:58.759 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:18:58.799 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-10T18:18:59.102 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-10T18:18:59.118 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-10T18:18:59.160 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-10T18:18:59.488 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-scipy. 2024-09-10T18:18:59.505 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-10T18:18:59.546 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-10T18:19:00.307 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-decorator. 2024-09-10T18:19:00.323 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-10T18:19:00.364 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-10T18:19:00.692 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-10T18:19:00.713 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-10T18:19:00.750 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-10T18:19:00.875 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-10T18:19:00.893 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-10T18:19:00.932 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:01.078 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-scipy. 2024-09-10T18:19:01.094 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-10T18:19:01.136 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-10T18:19:01.461 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T18:19:01.480 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-10T18:19:01.544 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-10T18:19:01.881 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T18:19:01.898 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-10T18:19:01.939 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:02.521 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-10T18:19:02.539 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-10T18:19:02.579 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:02.664 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T18:19:02.684 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:19:02.722 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:03.149 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T18:19:03.166 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-10T18:19:03.207 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-10T18:19:03.610 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T18:19:03.627 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-10T18:19:03.668 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:03.834 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T18:19:03.850 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-10T18:19:03.891 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-10T18:19:04.194 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T18:19:04.213 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-10T18:19:04.253 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-10T18:19:04.384 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T18:19:04.400 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:19:04.442 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:04.606 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T18:19:04.623 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-10T18:19:04.672 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-10T18:19:04.984 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T18:19:05.001 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-10T18:19:05.041 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-10T18:19:05.361 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T18:19:05.379 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-10T18:19:05.419 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-10T18:19:05.670 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T18:19:05.688 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-10T18:19:05.729 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-10T18:19:05.756 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-blinker. 2024-09-10T18:19:05.776 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-10T18:19:05.822 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-10T18:19:06.125 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-10T18:19:06.142 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-10T18:19:06.350 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-10T18:19:06.381 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T18:19:06.401 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-10T18:19:06.440 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-10T18:19:06.767 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T18:19:06.786 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-10T18:19:06.825 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-10T18:19:06.863 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T18:19:06.883 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-10T18:19:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-10T18:19:07.161 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T18:19:07.178 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-10T18:19:07.219 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-10T18:19:07.249 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T18:19:07.268 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-10T18:19:07.307 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-10T18:19:07.548 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T18:19:07.566 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-10T18:19:07.606 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-10T18:19:07.618 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T18:19:07.635 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-10T18:19:07.676 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-10T18:19:07.909 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-blinker. 2024-09-10T18:19:07.929 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-10T18:19:07.967 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-10T18:19:08.295 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-10T18:19:08.309 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T18:19:08.312 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-10T18:19:08.328 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:19:08.359 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:08.362 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-10T18:19:08.670 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T18:19:08.687 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-10T18:19:08.698 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T18:19:08.718 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-10T18:19:08.728 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-10T18:19:08.757 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-10T18:19:09.067 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T18:19:09.069 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T18:19:09.084 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-10T18:19:09.085 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-10T18:19:09.127 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:09.151 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-10T18:19:09.413 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package jq. 2024-09-10T18:19:09.430 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-10T18:19:09.462 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T18:19:09.471 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:09.482 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-10T18:19:09.520 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-10T18:19:09.724 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package socat. 2024-09-10T18:19:09.740 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-10T18:19:09.782 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-10T18:19:10.119 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T18:19:10.139 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-10T18:19:10.177 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-10T18:19:10.186 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T18:19:10.203 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:19:10.277 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:10.533 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-test. 2024-09-10T18:19:10.550 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:10.591 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-test (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:10.597 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T18:19:10.615 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-10T18:19:10.657 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-10T18:19:11.036 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T18:19:11.057 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-10T18:19:11.094 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:11.381 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package jq. 2024-09-10T18:19:11.401 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-10T18:19:11.439 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:11.675 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package socat. 2024-09-10T18:19:11.691 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-10T18:19:11.724 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-10T18:19:12.070 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T18:19:12.087 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-10T18:19:12.128 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-10T18:19:12.483 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-test. 2024-09-10T18:19:12.499 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:12.541 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-test (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:14.995 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T18:19:15.014 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:19:15.053 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking ceph-volume (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:15.346 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-10T18:19:15.362 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-10T18:19:15.404 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:15.916 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T18:19:15.933 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:15.974 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libcephfs-dev (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:16.363 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T18:19:16.379 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-09-10T18:19:16.438 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-09-10T18:19:16.809 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-10T18:19:16.826 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-10T18:19:16.867 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:17.151 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T18:19:17.167 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1606-g5be57395-1focal_all.deb ... 2024-09-10T18:19:17.208 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-volume (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:17.547 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-10T18:19:17.566 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-10T18:19:17.605 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:18.159 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T18:19:18.178 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:18.219 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs-dev (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:18.512 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-10T18:19:18.534 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-10T18:19:18.578 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-10T18:19:18.639 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T18:19:18.656 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-09-10T18:19:18.714 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-09-10T18:19:18.890 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-10T18:19:18.909 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-10T18:19:18.948 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-10T18:19:19.061 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-10T18:19:19.079 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-10T18:19:19.119 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:19.193 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-attr. 2024-09-10T18:19:19.213 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-10T18:19:19.242 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-10T18:19:19.620 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-webencodings. 2024-09-10T18:19:19.638 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-10T18:19:19.681 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-10T18:19:19.981 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-html5lib. 2024-09-10T18:19:19.999 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-10T18:19:20.039 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-10T18:19:20.418 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-10T18:19:20.434 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-10T18:19:20.484 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-10T18:19:20.763 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-10T18:19:20.782 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-10T18:19:20.821 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-10T18:19:20.929 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T18:19:20.946 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T18:19:20.987 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T18:19:21.149 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-10T18:19:21.165 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-10T18:19:21.232 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-10T18:19:21.332 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T18:19:21.350 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-10T18:19:21.390 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-10T18:19:21.552 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-attr. 2024-09-10T18:19:21.571 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-10T18:19:21.601 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-10T18:19:21.702 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T18:19:21.721 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-10T18:19:21.759 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-10T18:19:21.937 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webencodings. 2024-09-10T18:19:21.956 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-10T18:19:21.995 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-10T18:19:22.088 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-py. 2024-09-10T18:19:22.108 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-10T18:19:22.162 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-10T18:19:22.282 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-html5lib. 2024-09-10T18:19:22.302 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-10T18:19:22.365 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-10T18:19:22.482 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T18:19:22.501 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-10T18:19:22.540 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-10T18:19:22.693 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-10T18:19:22.714 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-10T18:19:22.751 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-10T18:19:22.852 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T18:19:22.870 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-10T18:19:22.910 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-10T18:19:23.238 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T18:19:23.254 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-10T18:19:23.262 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T18:19:23.282 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T18:19:23.295 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-10T18:19:23.320 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T18:19:23.616 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T18:19:23.636 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-10T18:19:23.666 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T18:19:23.673 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-10T18:19:23.686 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-10T18:19:23.724 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-10T18:19:23.984 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T18:19:24.002 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-10T18:19:24.043 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-10T18:19:24.069 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T18:19:24.087 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-10T18:19:24.126 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-10T18:19:24.430 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-py. 2024-09-10T18:19:24.447 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-10T18:19:24.521 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-10T18:19:24.747 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package radosgw. 2024-09-10T18:19:24.768 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../108-radosgw_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:24.805 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking radosgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:24.824 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T18:19:24.841 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-10T18:19:24.882 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-10T18:19:25.202 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T18:19:25.222 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-10T18:19:25.260 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-10T18:19:25.605 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T18:19:25.623 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-10T18:19:25.662 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-10T18:19:25.999 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T18:19:26.017 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-10T18:19:26.057 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-10T18:19:26.386 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T18:19:26.402 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-10T18:19:26.443 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-10T18:19:26.528 INFO:teuthology.orchestra.run.smithi100.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T18:19:26.546 INFO:teuthology.orchestra.run.smithi100.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:26.586 INFO:teuthology.orchestra.run.smithi100.stdout:Unpacking rbd-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:26.904 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-10T18:19:27.189 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package radosgw. 2024-09-10T18:19:27.207 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-10T18:19:27.210 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../108-radosgw_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:27.247 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking radosgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:27.333 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-10T18:19:27.462 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-09-10T18:19:27.655 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:28.233 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user cephadm....done 2024-09-10T18:19:28.396 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:28.514 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-10T18:19:28.841 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T18:19:29.070 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T18:19:29.087 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1606-g5be57395-1focal_amd64.deb ... 2024-09-10T18:19:29.128 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking rbd-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:29.142 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:29.268 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-10T18:19:29.394 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-10T18:19:29.416 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-10T18:19:29.729 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-10T18:19:29.733 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-10T18:19:29.860 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-10T18:19:29.986 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-09-10T18:19:30.156 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-10T18:19:30.204 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:30.415 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-10T18:19:30.718 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-10T18:19:30.766 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user cephadm....done 2024-09-10T18:19:30.929 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:31.002 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-10T18:19:31.055 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-10T18:19:31.304 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-argparse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:31.423 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T18:19:31.621 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:31.716 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-10T18:19:31.739 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-10T18:19:31.850 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-10T18:19:31.976 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-10T18:19:32.007 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-10T18:19:32.274 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-10T18:19:32.303 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-10T18:19:32.417 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-10T18:19:32.714 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-10T18:19:32.793 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-10T18:19:33.041 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-10T18:19:33.086 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-10T18:19:33.350 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-10T18:19:33.379 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-10T18:19:33.643 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-10T18:19:33.730 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-10T18:19:33.936 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-argparse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:34.006 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-10T18:19:34.229 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:34.332 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-10T18:19:34.346 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-10T18:19:34.633 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-10T18:19:34.672 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-10T18:19:34.926 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-10T18:19:34.956 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-10T18:19:35.052 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-10T18:19:35.082 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-10T18:19:35.203 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-10T18:19:35.492 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-10T18:19:35.496 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-10T18:19:35.768 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-10T18:19:35.889 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-10T18:19:36.061 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-10T18:19:36.257 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-10T18:19:36.387 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-10T18:19:36.625 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-10T18:19:36.663 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-10T18:19:36.751 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:37.078 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:19:37.389 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-10T18:19:37.616 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-10T18:19:37.757 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-10T18:19:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-10T18:19:38.088 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-10T18:19:38.214 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-10T18:19:38.218 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-10T18:19:38.340 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-10T18:19:38.368 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-10T18:19:38.633 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-10T18:19:38.653 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-10T18:19:38.942 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:19:39.046 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-10T18:19:39.068 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-10T18:19:39.195 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-10T18:19:39.406 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-10T18:19:39.521 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-10T18:19:39.791 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-10T18:19:39.814 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-10T18:19:39.942 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:40.106 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-10T18:19:40.232 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T18:19:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:19:40.436 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-10T18:19:40.500 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-10T18:19:40.779 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-10T18:19:40.792 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-10T18:19:40.905 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-10T18:19:41.031 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-10T18:19:41.094 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-10T18:19:41.357 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-10T18:19:41.441 INFO:teuthology.orchestra.run.smithi100.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-10T18:19:41.696 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-10T18:19:41.708 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:19:41.822 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-10T18:19:41.834 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-10T18:19:41.961 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-10T18:19:42.157 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-10T18:19:42.287 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-10T18:19:42.500 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-10T18:19:42.571 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-10T18:19:42.651 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-10T18:19:42.777 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-10T18:19:42.856 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-10T18:19:42.982 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T18:19:43.111 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-10T18:19:43.275 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-10T18:19:43.404 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librados2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:43.547 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-10T18:19:43.609 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-10T18:19:43.918 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-10T18:19:44.341 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-10T18:19:44.571 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-10T18:19:44.689 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-10T18:19:45.011 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:45.015 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-10T18:19:45.153 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:45.279 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-10T18:19:45.374 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-10T18:19:45.534 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-10T18:19:45.660 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-10T18:19:45.689 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-10T18:19:45.978 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-10T18:19:46.007 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-10T18:19:46.270 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librados2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:46.291 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:19:46.396 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-10T18:19:46.408 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-10T18:19:46.743 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-10T18:19:47.213 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libradosstriper1 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:47.339 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-10T18:19:47.656 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:47.844 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:47.969 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:48.095 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-10T18:19:48.464 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-10T18:19:48.662 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librbd1 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:48.781 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-10T18:19:48.787 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-10T18:19:48.914 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-10T18:19:49.058 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-10T18:19:49.065 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:49.177 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-10T18:19:49.190 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-10T18:19:49.467 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-10T18:19:49.503 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:49.593 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-10T18:19:49.629 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-10T18:19:49.894 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-modules-core (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:49.956 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libradosstriper1 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:50.053 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:50.082 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-10T18:19:50.275 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T18:19:50.275 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T18:19:50.425 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-10T18:19:50.738 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up libcephfs-dev (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:50.966 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-10T18:19:51.234 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T18:19:51.475 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librbd1 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:51.544 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-10T18:19:51.606 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-10T18:19:51.732 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-10T18:19:51.858 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-10T18:19:51.905 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rados (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:51.976 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-10T18:19:52.031 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up librgw2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:52.157 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rbd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:52.283 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up rbd-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:52.302 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-10T18:19:52.400 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-10T18:19:52.428 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-10T18:19:52.705 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-10T18:19:52.721 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-modules-core (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:52.872 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:53.054 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-rgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:53.089 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T18:19:53.089 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T18:19:53.179 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-cephfs (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:53.305 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-10T18:19:53.540 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs-dev (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:53.777 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-10T18:19:54.062 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T18:19:54.200 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:54.371 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-10T18:19:54.728 INFO:teuthology.orchestra.run.smithi100.stdout:Adding group ceph....done 2024-09-10T18:19:54.765 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rados (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:54.891 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librgw2 (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:55.017 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rbd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:55.113 INFO:teuthology.orchestra.run.smithi100.stdout:Adding system user ceph....done 2024-09-10T18:19:55.143 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up rbd-fuse (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:55.280 INFO:teuthology.orchestra.run.smithi100.stdout:Setting system user ceph properties....done 2024-09-10T18:19:55.286 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-10T18:19:55.307 INFO:teuthology.orchestra.run.smithi100.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T18:19:55.454 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T18:19:55.596 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-10T18:19:55.777 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T18:19:55.898 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:56.024 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cephfs (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:56.149 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-10T18:19:56.226 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-test (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:56.361 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up radosgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:56.659 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:19:56.659 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:19:57.045 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-common (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:57.271 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-base (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:57.605 INFO:teuthology.orchestra.run.smithi151.stdout:Adding group ceph....done 2024-09-10T18:19:57.679 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T18:19:57.957 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user ceph....done 2024-09-10T18:19:58.159 INFO:teuthology.orchestra.run.smithi151.stdout:Setting system user ceph properties....done 2024-09-10T18:19:58.173 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mds (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:58.178 INFO:teuthology.orchestra.run.smithi151.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T18:19:58.321 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T18:19:58.429 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T18:19:58.429 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T18:19:58.633 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T18:19:58.883 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:59.088 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-test (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:59.111 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T18:19:59.111 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T18:19:59.239 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up radosgw (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:19:59.514 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:19:59.514 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:19:59.567 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-osd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:00.057 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T18:20:00.058 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T18:20:00.182 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-base (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:00.504 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:00.539 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T18:20:00.620 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:00.754 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mon (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:00.968 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mds (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:00.974 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T18:20:00.974 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T18:20:01.186 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T18:20:01.186 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T18:20:01.422 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:01.565 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:01.636 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:01.691 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-mgr-dashboard (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:01.817 INFO:teuthology.orchestra.run.smithi100.stdout:Setting up ceph-volume (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:01.863 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T18:20:01.863 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T18:20:01.944 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-10T18:20:02.120 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-10T18:20:02.304 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-osd (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:02.438 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-10T18:20:02.831 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T18:20:02.831 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T18:20:03.289 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:03.415 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:03.583 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mon (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:03.801 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T18:20:03.802 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T18:20:04.251 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-cephadm (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:04.394 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:04.520 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-dashboard (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:04.696 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-volume (17.2.7-1606-g5be57395-1focal) ... 2024-09-10T18:20:04.823 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-10T18:20:04.974 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-10T18:20:05.250 INFO:teuthology.orchestra.run.smithi100.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-10T18:20:05.267 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-10T18:20:05.867 INFO:teuthology.orchestra.run.smithi100.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T18:20:05.873 DEBUG:teuthology.parallel:result is None 2024-09-10T18:20:08.225 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-10T18:20:09.291 INFO:teuthology.orchestra.run.smithi151.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T18:20:09.297 DEBUG:teuthology.parallel:result is None 2024-09-10T18:20:09.298 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:20:09.443 DEBUG:teuthology.orchestra.run.smithi100:> dpkg-query -W -f '${Version}' ceph 2024-09-10T18:20:09.467 INFO:teuthology.orchestra.run.smithi100.stdout:17.2.7-1606-g5be57395-1focal 2024-09-10T18:20:09.468 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606-g5be57395-1focal 2024-09-10T18:20:09.468 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606-g5be57395-1focal is installed. 2024-09-10T18:20:09.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:20:09.612 DEBUG:teuthology.orchestra.run.smithi151:> dpkg-query -W -f '${Version}' ceph 2024-09-10T18:20:09.635 INFO:teuthology.orchestra.run.smithi151.stdout:17.2.7-1606-g5be57395-1focal 2024-09-10T18:20:09.635 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606-g5be57395-1focal 2024-09-10T18:20:09.635 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606-g5be57395-1focal is installed. 2024-09-10T18:20:09.637 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T18:20:09.637 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:20:09.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:20:09.653 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:20:09.653 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:20:09.692 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T18:20:09.693 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:20:09.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:20:09.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:20:09.722 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:20:09.722 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:20:09.738 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:20:09.795 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T18:20:09.796 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:20:09.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:20:09.811 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:20:09.869 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:20:09.869 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:20:09.884 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:20:09.899 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T18:20:09.899 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:20:09.899 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:20:09.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:20:09.924 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:20:09.924 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:20:09.956 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:20:09.965 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:20:10.057 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'root'} 2024-09-10T18:20:10.057 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:20:10.057 INFO:tasks.cephadm:Cluster fsid is 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:20:10.058 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:20:10.058 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.100', 'mon.c': '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]', 'mon.b': '172.21.15.151'} 2024-09-10T18:20:10.058 INFO:tasks.cephadm:First mon is mon.a on smithi100 2024-09-10T18:20:10.058 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:20:10.058 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:20:10.058 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-09-10T18:20:10.076 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-09-10T18:20:10.092 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref 5be57395ea235535910512ae73ba8e7ff25fe588)... 2024-09-10T18:20:10.093 DEBUG:teuthology.orchestra.run.smithi100:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:20:10.124 INFO:teuthology.orchestra.run.smithi100.stderr:Cloning into 'testrepo'... 2024-09-10T18:21:11.446 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-rw-r-- 1 ubuntu ubuntu 378585 Sep 10 18:21 /home/ubuntu/cephtest/cephadm 2024-09-10T18:21:11.447 DEBUG:teuthology.orchestra.run.smithi151:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:21:11.455 INFO:teuthology.orchestra.run.smithi151.stderr:Cloning into 'testrepo'... 2024-09-10T18:22:11.268 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-rw-r-- 1 ubuntu ubuntu 378585 Sep 10 18:22 /home/ubuntu/cephtest/cephadm 2024-09-10T18:22:11.269 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:22:11.279 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:22:11.306 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:22:11.306 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:22:11.309 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:22:11.576 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:22:11.592 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:22:59.913 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-10T18:22:59.913 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:22:59.913 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:22:59.913 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-09-10T18:22:59.913 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:22:59.913 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-10T18:22:59.914 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-10T18:23:02.843 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-10T18:23:02.843 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:23:02.843 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:23:02.843 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-09-10T18:23:02.843 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:23:02.844 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-10T18:23:02.844 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-10T18:23:02.868 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-09-10T18:23:02.885 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-09-10T18:23:02.899 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-09-10T18:23:02.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-09-10T18:23:02.928 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:23:02.929 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:23:02.929 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:23:02.929 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-10T18:23:02.930 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:23:02.931 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:23:02.931 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:23:02.937 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 505ebbde-6fa1-11ef-bcea-c7b262605968 mon election default strategy = 3 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-10T18:23:02.938 DEBUG:teuthology.orchestra.run.smithi100:mon.a> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:23:02.982 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:23:02.985 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:23:02.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.100 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:23:03.149 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:23:03.149 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', '505ebbde-6fa1-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.100', '--skip-admin-label'] 2024-09-10T18:23:03.150 INFO:teuthology.orchestra.run.smithi100.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:23:03.150 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying podman|docker is present... 2024-09-10T18:23:03.150 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying lvm2 is present... 2024-09-10T18:23:03.150 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying time synchronization is in place... 2024-09-10T18:23:03.156 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:23:03.157 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:23:03.163 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:23:03.163 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.172 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-09-10T18:23:03.172 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-09-10T18:23:03.179 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-09-10T18:23:03.179 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.185 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-09-10T18:23:03.185 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout masked 2024-09-10T18:23:03.191 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-09-10T18:23:03.192 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.199 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-09-10T18:23:03.199 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-09-10T18:23:03.206 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-09-10T18:23:03.206 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.214 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-09-10T18:23:03.222 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-09-10T18:23:03.222 INFO:teuthology.orchestra.run.smithi100.stdout:Unit ntp.service is enabled and running 2024-09-10T18:23:03.223 INFO:teuthology.orchestra.run.smithi100.stdout:Repeating the final host check... 2024-09-10T18:23:03.223 INFO:teuthology.orchestra.run.smithi100.stdout:docker (/usr/bin/docker) is present 2024-09-10T18:23:03.223 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl is present 2024-09-10T18:23:03.223 INFO:teuthology.orchestra.run.smithi100.stdout:lvcreate is present 2024-09-10T18:23:03.229 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:23:03.230 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:23:03.236 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:23:03.236 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.242 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-09-10T18:23:03.242 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-09-10T18:23:03.249 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-09-10T18:23:03.249 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.256 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-09-10T18:23:03.256 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout masked 2024-09-10T18:23:03.263 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-09-10T18:23:03.263 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.269 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-09-10T18:23:03.269 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-09-10T18:23:03.276 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-09-10T18:23:03.276 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-09-10T18:23:03.284 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-09-10T18:23:03.291 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-09-10T18:23:03.292 INFO:teuthology.orchestra.run.smithi100.stdout:Unit ntp.service is enabled and running 2024-09-10T18:23:03.292 INFO:teuthology.orchestra.run.smithi100.stdout:Host looks OK 2024-09-10T18:23:03.292 INFO:teuthology.orchestra.run.smithi100.stdout:Cluster fsid: 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:03.292 INFO:teuthology.orchestra.run.smithi100.stdout:Acquiring lock 139738175314528 on /run/cephadm/505ebbde-6fa1-11ef-bcea-c7b262605968.lock 2024-09-10T18:23:03.292 INFO:teuthology.orchestra.run.smithi100.stdout:Lock 139738175314528 acquired on /run/cephadm/505ebbde-6fa1-11ef-bcea-c7b262605968.lock 2024-09-10T18:23:03.292 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 3300 ... 2024-09-10T18:23:03.292 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 6789 ... 2024-09-10T18:23:03.293 INFO:teuthology.orchestra.run.smithi100.stdout:Base mon IP(s) is [172.21.15.100:3300, 172.21.15.100:6789], mon addrv is [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-09-10T18:23:03.299 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 2024-09-10T18:23:03.299 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.100 metric 100 2024-09-10T18:23:03.299 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-09-10T18:23:03.299 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.100 2024-09-10T18:23:03.299 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.100 metric 100 2024-09-10T18:23:03.304 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:23:03.304 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-09-10T18:23:03.304 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1787sec pref medium 2024-09-10T18:23:03.308 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:23:03.308 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:23:03.308 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:23:03.308 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T18:23:03.308 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-09-10T18:23:03.308 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:23:03.309 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-09-10T18:23:03.309 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-09-10T18:23:03.309 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.15.254/32` 2024-09-10T18:23:03.309 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.15.254/32` 2024-09-10T18:23:03.309 INFO:teuthology.orchestra.run.smithi100.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20', '172.21.15.254/32', '172.21.15.254/32'] 2024-09-10T18:23:03.310 INFO:teuthology.orchestra.run.smithi100.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:23:03.310 INFO:teuthology.orchestra.run.smithi100.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:23:03.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:23:03.456 INFO:journalctl@ceph.mgr.y.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:23:03.614 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout 5be57395ea235535910512ae73ba8e7ff25fe588: Pulling from ceph-ci/ceph 2024-09-10T18:23:03.614 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout Digest: sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:23:03.614 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout Status: Image is up to date for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:23:03.614 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/docker: stdout quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:23:05.248 INFO:teuthology.orchestra.run.smithi100.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:23:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:23:05.249 INFO:teuthology.orchestra.run.smithi100.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:23:06.796 INFO:teuthology.orchestra.run.smithi100.stdout:stat: stdout 167 167 2024-09-10T18:23:06.796 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial keys... 2024-09-10T18:23:08.293 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCLjuBmho2sKRAA+mu4wuPkrbGj8eoYhX9x4A== 2024-09-10T18:23:09.816 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCNjuBmEwrqDhAAJyQmve43b/x1NZPWq9snrw== 2024-09-10T18:23:11.288 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQCOjuBmXI7YKxAAbktIYy15hrhMSpvO0xKj4w== 2024-09-10T18:23:11.289 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial monmap... 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool for a [v2:172.21.15.100:3300,v1:172.21.15.100:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = octopus 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: set fsid to 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:23:12.937 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mon... 2024-09-10T18:23:14.502 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 0 set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:23:14.502 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 1 imported monmap: 2024-09-10T18:23:14.502 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr epoch 0 2024-09-10T18:23:14.502 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:14.502 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr last_changed 2024-09-10T18:23:12.325186+0000 2024-09-10T18:23:14.502 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr created 2024-09-10T18:23:12.325186+0000 2024-09-10T18:23:14.502 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr min_mon_release 15 (octopus) 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr election_strategy: 1 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 0 /usr/bin/ceph-mon: set fsid to 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: RocksDB version: 6.15.5 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Compile date Sep 9 2024 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: DB SUMMARY 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.506 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: DB Session ID: MLFKN7OJDHQXWRFSQJFJ 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.error_if_exists: 0 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.create_if_missing: 1 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.paranoid_checks: 1 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.env: 0x56176db99440 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.fs: Posix File System 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.info_log: 0x5617703d4660 2024-09-10T18:23:14.507 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.statistics: (nil) 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.use_fsync: 0 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_log_file_size: 0 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.allow_fallocate: 1 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:23:14.508 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:23:14.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.use_direct_reads: 0 2024-09-10T18:23:14.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:23:14.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:23:14.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.db_log_dir: 2024-09-10T18:23:14.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.write_buffer_manager: 0x56177041de90 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:23:14.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.unordered_write: 0 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.row_cache: None 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.wal_filter: None 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:23:14.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.preserve_deletes: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.two_write_queues: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.atomic_flush: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.log_readahead_size: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:23:14.534 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_background_jobs: 2 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_background_compactions: -1 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_subcompactions: 1 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:23:14.535 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_open_files: -1 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Options.max_background_flushes: -1 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Compression algorithms supported: 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kZSTD supported: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kXpressCompression supported: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kBZip2Compression supported: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kLZ4Compression supported: 1 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kZlibCompression supported: 1 2024-09-10T18:23:14.536 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: kSnappyCompression supported: 1 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.980+0000 7f6e3fdb9900 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: [db/db_impl/db_impl_open.cc:281] Creating manifest 1 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.merge_operator: 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_filter: None 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:23:14.537 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561770374ee0) 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks: 1 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr pin_top_level_index_and_filter: 1 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr index_type: 0 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr data_block_index_type: 0 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr index_shortening: 1 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr hash_index_allow_collision: 1 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr checksum: 1 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr no_block_cache: 0 2024-09-10T18:23:14.538 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache: 0x5617703ad1f0 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache_name: BinnedLRUCache 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache_options: 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr capacity : 536870912 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr num_shard_bits : 4 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr strict_capacity_limit : 0 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr high_pri_pool_ratio: 0.000 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_cache_compressed: (nil) 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr persistent_cache: (nil) 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_size: 4096 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_size_deviation: 10 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_restart_interval: 16 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr index_block_restart_interval: 1 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr metadata_block_size: 4096 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr partition_filters: 0 2024-09-10T18:23:14.539 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr use_delta_encoding: 1 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr whole_key_filtering: 1 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr verify_compression: 0 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr read_amp_bytes_per_bit: 0 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr format_version: 4 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr enable_index_compression: 1 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr block_align: 0 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression: NoCompression 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:23:14.540 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.num_levels: 7 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:23:14.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:23:14.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:23:14.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:23:14.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.inplace_update_support: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.bloom_locality: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.max_successive_merges: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.ttl: 2592000 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:23:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.enable_blob_files: false 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.min_blob_size: 0 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.984+0000 7f6e3fdb9900 4 rocksdb: [db/version_set.cc:4082] Creating manifest 3 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.988+0000 7f6e3fdb9900 4 rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5617703de700 2024-09-10T18:23:14.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.988+0000 7f6e3fdb9900 4 rocksdb: DB pointer 0x5617703b0000 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.988+0000 7f6e37547640 4 rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.988+0000 7f6e37547640 4 rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** DB Stats ** 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:14.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:14.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:14.549 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.992+0000 7f6e3fdb9900 4 rocksdb: [db/db_impl/db_impl.cc:446] Shutdown: canceling all background work 2024-09-10T18:23:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.992+0000 7f6e3fdb9900 4 rocksdb: [db/db_impl/db_impl.cc:625] Shutdown complete 2024-09-10T18:23:14.551 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-mon: stderr debug 2024-09-10T18:23:13.992+0000 7f6e3fdb9900 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-09-10T18:23:14.551 INFO:teuthology.orchestra.run.smithi100.stdout:create mon.a on 2024-09-10T18:23:14.742 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-09-10T18:23:14.939 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-09-10T18:23:15.116 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968.target -> /etc/systemd/system/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968.target. 2024-09-10T18:23:15.117 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968.target -> /etc/systemd/system/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968.target. 2024-09-10T18:23:15.361 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:23:15.361 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:23:15.563 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968.target.wants/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a.service -> /etc/systemd/system/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@.service. 2024-09-10T18:23:15.576 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-09-10T18:23:15.576 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:23:15.577 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon to start... 2024-09-10T18:23:15.577 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon... 2024-09-10T18:23:15.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:15 smithi100 systemd[1]: Started Ceph mon.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:23:17.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:17 smithi100 bash[18096]: cluster 2024-09-10T18:23:17.238183+0000 mon.a (mon.0) 0 : cluster [INF] mkfs 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:17.452 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:17 smithi100 bash[18096]: cluster 2024-09-10T18:23:17.231064+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:23:18.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:23:18.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout id: 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:18.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:23:18.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:18.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.188847s) 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:23:18.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:23:18.033 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:18.033 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965c982640 1 Processor -- start 2024-09-10T18:23:18.033 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965c982640 1 -- start start 2024-09-10T18:23:18.033 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965c982640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658075b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:18.033 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965c982640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658077870 con 0x7f9658077460 2024-09-10T18:23:18.033 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658075b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658075b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33030/0 (socket says 172.21.15.100:33030) 2024-09-10T18:23:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965659c640 1 -- 172.21.15.100:0/814476532 learned_addr learned my addr 172.21.15.100:0/814476532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.284+0000 7f965659c640 1 -- 172.21.15.100:0/814476532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580779b0 con 0x7f9658077460 2024-09-10T18:23:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.288+0000 7f965659c640 1 --2- 172.21.15.100:0/814476532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658075b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f9648009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6cb5800c50bdb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.288+0000 7f965559a640 1 -- 172.21.15.100:0/814476532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9648031400 con 0x7f9658077460 2024-09-10T18:23:18.037 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.288+0000 7f965559a640 1 -- 172.21.15.100:0/814476532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964802d050 con 0x7f9658077460 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.288+0000 7f965559a640 1 -- 172.21.15.100:0/814476532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96480315c0 con 0x7f9658077460 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.288+0000 7f965c982640 1 -- 172.21.15.100:0/814476532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 msgr2=0x7f9658075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.288+0000 7f965c982640 1 --2- 172.21.15.100:0/814476532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658075b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f9648009840 comp rx=0 tx=0).stop 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.288+0000 7f965c982640 1 -- 172.21.15.100:0/814476532 shutdown_connections 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 --2- 172.21.15.100:0/814476532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658075b20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 -- 172.21.15.100:0/814476532 >> 172.21.15.100:0/814476532 conn(0x7f96580fbf30 msgr2=0x7f96580fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 -- 172.21.15.100:0/814476532 shutdown_connections 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 -- 172.21.15.100:0/814476532 wait complete. 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 Processor -- start 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 -- start start 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658110d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9658111260 con 0x7f9658077460 2024-09-10T18:23:18.038 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658110d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658110d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33036/0 (socket says 172.21.15.100:33036) 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965659c640 1 -- 172.21.15.100:0/3348537117 learned_addr learned my addr 172.21.15.100:0/3348537117 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965659c640 1 -- 172.21.15.100:0/3348537117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648009ae0 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965659c640 1 --2- 172.21.15.100:0/3348537117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658110d20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f964802ac70 tx=0x7f9648036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f964effd640 1 -- 172.21.15.100:0/3348537117 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9648031e00 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f964effd640 1 -- 172.21.15.100:0/3348537117 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9648007a10 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9658111460 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f964effd640 1 -- 172.21.15.100:0/3348537117 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9648007d00 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9658111ff0 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f964effd640 1 -- 172.21.15.100:0/3348537117 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f9648038070 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.292+0000 7f964effd640 1 -- 172.21.15.100:0/3348537117 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9648045ac0 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.296+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9658111930 con 0x7f9658077460 2024-09-10T18:23:18.039 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.296+0000 7f964effd640 1 -- 172.21.15.100:0/3348537117 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f9658111930 con 0x7f9658077460 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.420+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f965804b720 con 0x7f9658077460 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f964effd640 1 -- 172.21.15.100:0/3348537117 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f9648040070 con 0x7f9658077460 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 msgr2=0x7f9658110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f965c982640 1 --2- 172.21.15.100:0/3348537117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658110d20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f964802ac70 tx=0x7f9648036000 comp rx=0 tx=0).stop 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 shutdown_connections 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f965c982640 1 --2- 172.21.15.100:0/3348537117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658077460 0x7f9658110d20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 >> 172.21.15.100:0/3348537117 conn(0x7f96580fbf30 msgr2=0x7f9658107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 shutdown_connections 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:17.424+0000 7f965c982640 1 -- 172.21.15.100:0/3348537117 wait complete. 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:mon is available 2024-09-10T18:23:18.040 INFO:teuthology.orchestra.run.smithi100.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:23:18.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:18 smithi100 bash[18096]: cluster 2024-09-10T18:23:17.236426+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:23:18.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:18 smithi100 bash[18096]: cluster 2024-09-10T18:23:17.236693+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:23:18.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:18 smithi100 bash[18096]: cluster 2024-09-10T18:23:17.240634+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-09-10T18:23:18.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:18 smithi100 bash[18096]: cluster 2024-09-10T18:23:17.244764+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:23:18.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:18 smithi100 bash[18096]: cluster 2024-09-10T18:23:17.245708+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-09-10T18:23:18.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:18 smithi100 bash[18096]: audit 2024-09-10T18:23:17.425219+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.100:0/3348537117' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T18:23:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:19 smithi100 bash[18096]: audit 2024-09-10T18:23:19.327696+0000 mon.a (mon.0) 8 : audit [INF] from='client.? 172.21.15.100:0/449409682' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:23:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:19 smithi100 bash[18096]: audit 2024-09-10T18:23:19.331288+0000 mon.a (mon.0) 9 : audit [INF] from='client.? 172.21.15.100:0/449409682' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:23:19.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:19.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:23:19.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:19.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:23:19.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6f277640 1 Processor -- start 2024-09-10T18:23:19.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6f277640 1 -- start start 2024-09-10T18:23:19.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6f277640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:19.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6f277640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68103160 con 0x7ffb68102810 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6d013640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6d013640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33038/0 (socket says 172.21.15.100:33038) 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6d013640 1 -- 172.21.15.100:0/106048692 learned_addr learned my addr 172.21.15.100:0/106048692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.192+0000 7ffb6d013640 1 -- 172.21.15.100:0/106048692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb681032a0 con 0x7ffb68102810 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6d013640 1 --2- 172.21.15.100:0/106048692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68102c20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009840 tx=0x7ffb5c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd687ae73744165 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb67fff640 1 -- 172.21.15.100:0/106048692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c02add0 con 0x7ffb68102810 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb67fff640 1 -- 172.21.15.100:0/106048692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c032440 con 0x7ffb68102810 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb67fff640 1 -- 172.21.15.100:0/106048692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c032750 con 0x7ffb68102810 2024-09-10T18:23:19.960 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 -- 172.21.15.100:0/106048692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 msgr2=0x7ffb68102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 --2- 172.21.15.100:0/106048692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68102c20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009840 tx=0x7ffb5c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 -- 172.21.15.100:0/106048692 shutdown_connections 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 --2- 172.21.15.100:0/106048692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68102c20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 -- 172.21.15.100:0/106048692 >> 172.21.15.100:0/106048692 conn(0x7ffb68075de0 msgr2=0x7ffb68076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 -- 172.21.15.100:0/106048692 shutdown_connections 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 -- 172.21.15.100:0/106048692 wait complete. 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 Processor -- start 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 -- start start 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68110e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.196+0000 7ffb6f277640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681113a0 con 0x7ffb68102810 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6d013640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68110e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6d013640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68110e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33042/0 (socket says 172.21.15.100:33042) 2024-09-10T18:23:19.961 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6d013640 1 -- 172.21.15.100:0/449409682 learned_addr learned my addr 172.21.15.100:0/449409682 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6d013640 1 -- 172.21.15.100:0/449409682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c009450 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6d013640 1 --2- 172.21.15.100:0/449409682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68110e60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c02a700 tx=0x7ffb5c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c031a80 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb681115a0 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c031be0 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb5c03b400 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb68111930 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ffb5c039040 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ffb5c041030 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.200+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6804b720 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.204+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ffb5c037070 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.324+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ffb68103b00 con 0x7ffb68102810 2024-09-10T18:23:19.962 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.328+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 7 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7ffb5c0467d0 con 0x7ffb68102810 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.328+0000 7ffb65ffb640 1 -- 172.21.15.100:0/449409682 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+376 (secure 0 0 0) 0x7ffb5c055020 con 0x7ffb68102810 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.328+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 msgr2=0x7ffb68110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.332+0000 7ffb6f277640 1 --2- 172.21.15.100:0/449409682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68110e60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c02a700 tx=0x7ffb5c032000 comp rx=0 tx=0).stop 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.332+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 shutdown_connections 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.332+0000 7ffb6f277640 1 --2- 172.21.15.100:0/449409682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb68102810 0x7ffb68110e60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.332+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 >> 172.21.15.100:0/449409682 conn(0x7ffb68075de0 msgr2=0x7ffb68107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.332+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 shutdown_connections 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:19.332+0000 7ffb6f277640 1 -- 172.21.15.100:0/449409682 wait complete. 2024-09-10T18:23:19.963 INFO:teuthology.orchestra.run.smithi100.stdout:Generating new minimal ceph.conf... 2024-09-10T18:23:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:21 smithi100 bash[18096]: audit 2024-09-10T18:23:21.116086+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.100:0/203563031' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:23:21.747 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe6284de640 1 Processor -- start 2024-09-10T18:23:21.747 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe6284de640 1 -- start start 2024-09-10T18:23:21.747 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe6284de640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:21.747 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe6284de640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201031e0 con 0x7fe620102890 2024-09-10T18:23:21.747 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe62627a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:21.747 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe62627a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38978/0 (socket says 172.21.15.100:38978) 2024-09-10T18:23:21.747 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe62627a640 1 -- 172.21.15.100:0/2175038602 learned_addr learned my addr 172.21.15.100:0/2175038602 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe62627a640 1 -- 172.21.15.100:0/2175038602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe620103320 con 0x7fe620102890 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe62627a640 1 --2- 172.21.15.100:0/2175038602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620102ca0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe610009e50 tx=0x7fe610009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f971745d295b4d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe625278640 1 -- 172.21.15.100:0/2175038602 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe610031400 con 0x7fe620102890 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe625278640 1 -- 172.21.15.100:0/2175038602 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fe61002d050 con 0x7fe620102890 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.984+0000 7fe625278640 1 -- 172.21.15.100:0/2175038602 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6100076e0 con 0x7fe620102890 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- 172.21.15.100:0/2175038602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 msgr2=0x7fe620102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 --2- 172.21.15.100:0/2175038602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620102ca0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe610009e50 tx=0x7fe610009840 comp rx=0 tx=0).stop 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- 172.21.15.100:0/2175038602 shutdown_connections 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 --2- 172.21.15.100:0/2175038602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620102ca0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- 172.21.15.100:0/2175038602 >> 172.21.15.100:0/2175038602 conn(0x7fe620075f80 msgr2=0x7fe620076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- 172.21.15.100:0/2175038602 shutdown_connections 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- 172.21.15.100:0/2175038602 wait complete. 2024-09-10T18:23:21.748 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 Processor -- start 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- start start 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620111460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201119a0 con 0x7fe620102890 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe62627a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620111460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe62627a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620111460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38994/0 (socket says 172.21.15.100:38994) 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe62627a640 1 -- 172.21.15.100:0/203563031 learned_addr learned my addr 172.21.15.100:0/203563031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe62627a640 1 -- 172.21.15.100:0/203563031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610009ae0 con 0x7fe620102890 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe62627a640 1 --2- 172.21.15.100:0/203563031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620111460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6100070b0 tx=0x7fe610009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe61effd640 1 -- 172.21.15.100:0/203563031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe610007b60 con 0x7fe620102890 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe61effd640 1 -- 172.21.15.100:0/203563031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fe610007cc0 con 0x7fe620102890 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.988+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe620111ba0 con 0x7fe620102890 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.992+0000 7fe61effd640 1 -- 172.21.15.100:0/203563031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe610045e30 con 0x7fe620102890 2024-09-10T18:23:21.749 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.992+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe620112010 con 0x7fe620102890 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.992+0000 7fe61effd640 1 -- 172.21.15.100:0/203563031 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fe610038040 con 0x7fe620102890 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.992+0000 7fe61effd640 1 -- 172.21.15.100:0/203563031 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe610040030 con 0x7fe620102890 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.992+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62004b720 con 0x7fe620102890 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:20.996+0000 7fe61effd640 1 -- 172.21.15.100:0/203563031 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fe620112010 con 0x7fe620102890 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.112+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe6201127f0 con 0x7fe620102890 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.112+0000 7fe61effd640 1 -- 172.21.15.100:0/203563031 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fe610036070 con 0x7fe620102890 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.116+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 msgr2=0x7fe620111460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.116+0000 7fe6284de640 1 --2- 172.21.15.100:0/203563031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620111460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6100070b0 tx=0x7fe610009840 comp rx=0 tx=0).stop 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.116+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 shutdown_connections 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.116+0000 7fe6284de640 1 --2- 172.21.15.100:0/203563031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe620102890 0x7fe620111460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.116+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 >> 172.21.15.100:0/203563031 conn(0x7fe620075f80 msgr2=0x7fe620104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:21.750 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.116+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 shutdown_connections 2024-09-10T18:23:21.751 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:21.116+0000 7fe6284de640 1 -- 172.21.15.100:0/203563031 wait complete. 2024-09-10T18:23:21.751 INFO:teuthology.orchestra.run.smithi100.stdout:Restarting the monitor... 2024-09-10T18:23:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:21 smithi100 systemd[1]: Stopping Ceph mon.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:23:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:21 smithi100 bash[18096]: debug 2024-09-10T18:23:21.892+0000 7f67a8fe4640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-09-10T18:23:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:21 smithi100 bash[18096]: debug 2024-09-10T18:23:21.892+0000 7f67a8fe4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:23:22.640 INFO:teuthology.orchestra.run.smithi100.stdout:Setting public_network to 172.21.0.0/20,172.21.15.254/32 in mon config section 2024-09-10T18:23:22.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:22 smithi100 bash[18497]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:23:22.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:22 smithi100 bash[18562]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:23:22.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:22 smithi100 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a.service: Succeeded. 2024-09-10T18:23:22.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:22 smithi100 systemd[1]: Stopped Ceph mon.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:23:22.772 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:22 smithi100 systemd[1]: Started Ceph mon.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.036+0000 7f6cda251900 0 set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.036+0000 7f6cda251900 0 ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.036+0000 7f6cda251900 0 pidfile_write: ignore empty --pid-file 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 0 load: jerasure load: lrc 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: RocksDB version: 6.15.5 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Compile date Sep 9 2024 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: DB SUMMARY 2024-09-10T18:23:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: DB Session ID: 49XAIFL2RY06JPYSAS0Y 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: CURRENT file: CURRENT 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 85940 ; 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.error_if_exists: 0 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.create_if_missing: 0 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.paranoid_checks: 1 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.env: 0x561fa7d71440 2024-09-10T18:23:24.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.fs: Posix File System 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.info_log: 0x561fa8a6aee0 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.statistics: (nil) 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.use_fsync: 0 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_log_file_size: 0 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.allow_fallocate: 1 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:23:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.use_direct_reads: 0 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.db_log_dir: 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:23:24.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.write_buffer_manager: 0x561fa8b28780 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:23:24.299 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.unordered_write: 0 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.row_cache: None 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.wal_filter: None 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.preserve_deletes: 0 2024-09-10T18:23:24.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.two_write_queues: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.atomic_flush: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.log_readahead_size: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_background_jobs: 2 2024-09-10T18:23:24.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_background_compactions: -1 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_subcompactions: 1 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_open_files: -1 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:23:24.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_background_flushes: -1 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Compression algorithms supported: 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kZSTD supported: 0 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kXpressCompression supported: 0 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kBZip2Compression supported: 0 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kLZ4Compression supported: 1 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kZlibCompression supported: 1 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: kSnappyCompression supported: 1 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:23:24.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.merge_operator: 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_filter: None 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561fa8a13a38) 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: cache_index_and_filter_blocks: 1 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:23:24.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: pin_top_level_index_and_filter: 1 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: index_type: 0 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: data_block_index_type: 0 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: index_shortening: 1 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: hash_index_allow_collision: 1 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: checksum: 1 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: no_block_cache: 0 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_cache: 0x561fa8a47350 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_cache_name: BinnedLRUCache 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_cache_options: 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: capacity : 536870912 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: num_shard_bits : 4 2024-09-10T18:23:24.306 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: strict_capacity_limit : 0 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: high_pri_pool_ratio: 0.000 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_cache_compressed: (nil) 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: persistent_cache: (nil) 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_size: 4096 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_size_deviation: 10 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_restart_interval: 16 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: index_block_restart_interval: 1 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: metadata_block_size: 4096 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: partition_filters: 0 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: use_delta_encoding: 1 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: whole_key_filtering: 1 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: verify_compression: 0 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: read_amp_bytes_per_bit: 0 2024-09-10T18:23:24.307 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: format_version: 4 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: enable_index_compression: 1 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: block_align: 0 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression: NoCompression 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.num_levels: 7 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:23:24.308 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:23:24.309 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:23:24.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:23:24.311 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.inplace_update_support: 0 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.bloom_locality: 0 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.max_successive_merges: 0 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:23:24.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.ttl: 2592000 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.enable_blob_files: false 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.min_blob_size: 0 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:23:24.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1725992604045348, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.040+0000 7f6cda251900 4 rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cda251900 3 rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cda251900 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1725992604046464, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 82242, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 80488, "index_size": 216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10546, "raw_average_key_size": 47, "raw_value_size": 74592, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 221, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725992604, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "186f8812-2b09-42bd-aa21-e62eb613c966", "db_session_id": "49XAIFL2RY06JPYSAS0Y"}} 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cda251900 4 rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cda251900 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1725992604047390, "job": 1, "event": "recovery_finished"} 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cda251900 4 rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cda251900 4 rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x561fa8a7c700 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cda251900 4 rocksdb: DB pointer 0x561fa8a4a000 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cd001d640 4 rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.044+0000 7f6cd001d640 4 rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ** DB Stats ** 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:23:24.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ** Compaction Stats [default] ** 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: L0 2/0 82.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Sum 2/0 82.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ** Compaction Stats [default] ** 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:24.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 104.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Cumulative compaction: 0.00 GB write, 14.78 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Interval compaction: 0.00 GB write, 14.78 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ** Compaction Stats [default] ** 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:24.316 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: L0 2/0 82.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Sum 2/0 82.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ** Compaction Stats [default] ** 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 104.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:23:24.317 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Cumulative compaction: 0.00 GB write, 14.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 0 starting mon.a rank 0 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 1 mon.a@-1(???) e1 preinit fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 0 mon.a@-1(???).mds e1 new map 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 0 mon.a@-1(???).mds e1 print_map 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: e1 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: legacy client fscid: -1 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: No filesystems configured 2024-09-10T18:23:24.318 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: debug 2024-09-10T18:23:24.048+0000 7f6cda251900 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: cluster 2024-09-10T18:23:24.057395+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: cluster 2024-09-10T18:23:24.057452+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: cluster 2024-09-10T18:23:24.057964+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: cluster 2024-09-10T18:23:24.057999+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:23:24.319 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:24 smithi100 bash[18597]: cluster 2024-09-10T18:23:24.058482+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-09-10T18:23:24.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb4679640 1 Processor -- start 2024-09-10T18:23:24.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb4679640 1 -- start start 2024-09-10T18:23:24.916 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb4679640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac100a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:24.916 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb4679640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac100f90 con 0x7f5eac100640 2024-09-10T18:23:24.916 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb2415640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:24.916 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb2415640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39004/0 (socket says 172.21.15.100:39004) 2024-09-10T18:23:24.916 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb2415640 1 -- 172.21.15.100:0/1119050593 learned_addr learned my addr 172.21.15.100:0/1119050593 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.136+0000 7f5eb2415640 1 -- 172.21.15.100:0/1119050593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac1010d0 con 0x7f5eac100640 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb2415640 1 --2- 172.21.15.100:0/1119050593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac100a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009840 tx=0x7f5ea80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd878f8027a10c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb1413640 1 -- 172.21.15.100:0/1119050593 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ea802add0 con 0x7f5eac100640 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb1413640 1 -- 172.21.15.100:0/1119050593 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f5ea8032890 con 0x7f5eac100640 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb1413640 1 -- 172.21.15.100:0/1119050593 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ea80317a0 con 0x7f5eac100640 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb4679640 1 -- 172.21.15.100:0/1119050593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 msgr2=0x7f5eac100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb4679640 1 --2- 172.21.15.100:0/1119050593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac100a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009840 tx=0x7f5ea80091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb4679640 1 -- 172.21.15.100:0/1119050593 shutdown_connections 2024-09-10T18:23:24.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb4679640 1 --2- 172.21.15.100:0/1119050593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac100a50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.140+0000 7f5eb4679640 1 -- 172.21.15.100:0/1119050593 >> 172.21.15.100:0/1119050593 conn(0x7f5eac0fbe10 msgr2=0x7f5eac0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 -- 172.21.15.100:0/1119050593 shutdown_connections 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 -- 172.21.15.100:0/1119050593 wait complete. 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 Processor -- start 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 -- start start 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac10f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac10f7e0 con 0x7f5eac100640 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb2415640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac10f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb2415640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac10f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39014/0 (socket says 172.21.15.100:39014) 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb2415640 1 -- 172.21.15.100:0/2931524250 learned_addr learned my addr 172.21.15.100:0/2931524250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb2415640 1 -- 172.21.15.100:0/2931524250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8009450 con 0x7f5eac100640 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb2415640 1 --2- 172.21.15.100:0/2931524250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac10f2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80313e0 tx=0x7f5ea80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:24.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5ea2ffd640 1 -- 172.21.15.100:0/2931524250 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ea8031e70 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac10f9e0 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5ea2ffd640 1 -- 172.21.15.100:0/2931524250 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f5ea803c850 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5ea2ffd640 1 -- 172.21.15.100:0/2931524250 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ea803a540 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac10fe50 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.144+0000 7f5ea2ffd640 1 -- 172.21.15.100:0/2931524250 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5ea803aa50 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.148+0000 7f5ea2ffd640 1 -- 172.21.15.100:0/2931524250 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5ea8046c60 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.148+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac04b720 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.152+0000 7f5ea2ffd640 1 -- 172.21.15.100:0/2931524250 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5eac10fe50 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.268+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f5eac102370 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.276+0000 7f5ea2ffd640 1 -- 172.21.15.100:0/2931524250 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ea8037070 con 0x7f5eac100640 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.276+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 msgr2=0x7f5eac10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.276+0000 7f5eb4679640 1 --2- 172.21.15.100:0/2931524250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac10f2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80313e0 tx=0x7f5ea80091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:24.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.276+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 shutdown_connections 2024-09-10T18:23:24.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.276+0000 7f5eb4679640 1 --2- 172.21.15.100:0/2931524250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eac100640 0x7f5eac10f2a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:24.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.276+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 >> 172.21.15.100:0/2931524250 conn(0x7f5eac0fbe10 msgr2=0x7f5eac0fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:24.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.280+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 shutdown_connections 2024-09-10T18:23:24.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:24.280+0000 7f5eb4679640 1 -- 172.21.15.100:0/2931524250 wait complete. 2024-09-10T18:23:24.923 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:23:24.923 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:23:24.924 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mgr... 2024-09-10T18:23:24.924 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:23:24.924 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:23:25.136 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:23:25.136 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:23:25.333 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968.target.wants/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.y.service -> /etc/systemd/system/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@.service. 2024-09-10T18:23:25.345 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-09-10T18:23:25.345 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:23:25.346 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-09-10T18:23:25.346 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:23:25.346 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr to start... 2024-09-10T18:23:25.346 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr... 2024-09-10T18:23:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:25 smithi100 bash[18597]: audit 2024-09-10T18:23:24.278544+0000 mon.a (mon.0) 6 : audit [INF] from='client.? 172.21.15.100:0/2931524250' entity='client.admin' 2024-09-10T18:23:27.194 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:27 smithi100 bash[18863]: debug 2024-09-10T18:23:27.056+0000 7fbd8e002ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:23:27.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:27 smithi100 bash[18597]: audit 2024-09-10T18:23:27.172301+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.100:0/1138656315' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:23:27.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:27.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "505ebbde-6fa1-11ef-bcea-c7b262605968", 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:27.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:23:27.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:23:27.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:23:27.800 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:23:17.238597+0000", 2024-09-10T18:23:27.801 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e44f6640 1 Processor -- start 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e44f6640 1 -- start start 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e44f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc103170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e44f6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1036b0 con 0x7f44dc074c60 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e2292640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:27.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e2292640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39048/0 (socket says 172.21.15.100:39048) 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e2292640 1 -- 172.21.15.100:0/311262445 learned_addr learned my addr 172.21.15.100:0/311262445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.032+0000 7f44e2292640 1 -- 172.21.15.100:0/311262445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc1037f0 con 0x7f44dc074c60 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e2292640 1 --2- 172.21.15.100:0/311262445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc103170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009840 tx=0x7f44d80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29f70a5d8be9876 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e1290640 1 -- 172.21.15.100:0/311262445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44d802add0 con 0x7f44dc074c60 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e1290640 1 -- 172.21.15.100:0/311262445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f44d8032890 con 0x7f44dc074c60 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e1290640 1 -- 172.21.15.100:0/311262445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44d80317a0 con 0x7f44dc074c60 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 -- 172.21.15.100:0/311262445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 msgr2=0x7f44dc103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 --2- 172.21.15.100:0/311262445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc103170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009840 tx=0x7f44d80091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:27.803 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 -- 172.21.15.100:0/311262445 shutdown_connections 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 --2- 172.21.15.100:0/311262445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc103170 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 -- 172.21.15.100:0/311262445 >> 172.21.15.100:0/311262445 conn(0x7f44dc0fabc0 msgr2=0x7f44dc0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 -- 172.21.15.100:0/311262445 shutdown_connections 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 -- 172.21.15.100:0/311262445 wait complete. 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 Processor -- start 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 -- start start 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc1114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.036+0000 7f44e44f6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc111a30 con 0x7f44dc074c60 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e2292640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc1114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e2292640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc1114f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39054/0 (socket says 172.21.15.100:39054) 2024-09-10T18:23:27.804 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e2292640 1 -- 172.21.15.100:0/1138656315 learned_addr learned my addr 172.21.15.100:0/1138656315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e2292640 1 -- 172.21.15.100:0/1138656315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8009450 con 0x7f44dc074c60 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e2292640 1 --2- 172.21.15.100:0/1138656315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc1114f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f44d80313e0 tx=0x7f44d80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44d2ffd640 1 -- 172.21.15.100:0/1138656315 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44d8032cd0 con 0x7f44dc074c60 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc111c30 con 0x7f44dc074c60 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1120a0 con 0x7f44dc074c60 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44d2ffd640 1 -- 172.21.15.100:0/1138656315 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f44d80468b0 con 0x7f44dc074c60 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44d2ffd640 1 -- 172.21.15.100:0/1138656315 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44d803c400 con 0x7f44dc074c60 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44d2ffd640 1 -- 172.21.15.100:0/1138656315 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f44d803c8f0 con 0x7f44dc074c60 2024-09-10T18:23:27.805 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44d2ffd640 1 -- 172.21.15.100:0/1138656315 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f44d803aae0 con 0x7f44dc074c60 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.040+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c0005040 con 0x7f44dc074c60 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.044+0000 7f44d2ffd640 1 -- 172.21.15.100:0/1138656315 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f44dc1120a0 con 0x7f44dc074c60 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.168+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f44c00052f0 con 0x7f44dc074c60 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.168+0000 7f44d2ffd640 1 -- 172.21.15.100:0/1138656315 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f44d8055090 con 0x7f44dc074c60 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.172+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 msgr2=0x7f44dc1114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.172+0000 7f44e44f6640 1 --2- 172.21.15.100:0/1138656315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc1114f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f44d80313e0 tx=0x7f44d80091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.172+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 shutdown_connections 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.172+0000 7f44e44f6640 1 --2- 172.21.15.100:0/1138656315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44dc074c60 0x7f44dc1114f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.172+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 >> 172.21.15.100:0/1138656315 conn(0x7f44dc0fabc0 msgr2=0x7f44dc100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.172+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 shutdown_connections 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:27.172+0000 7f44e44f6640 1 -- 172.21.15.100:0/1138656315 wait complete. 2024-09-10T18:23:27.806 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:23:28.096 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:27 smithi100 bash[18863]: debug 2024-09-10T18:23:27.816+0000 7fbd8e002ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:23:28.096 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:27 smithi100 bash[18863]: debug 2024-09-10T18:23:27.880+0000 7fbd8e002ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:23:28.097 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:27 smithi100 bash[18863]: debug 2024-09-10T18:23:27.944+0000 7fbd8e002ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:23:28.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:28 smithi100 bash[18863]: debug 2024-09-10T18:23:28.092+0000 7fbd8e002ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:23:28.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:28 smithi100 bash[18863]: debug 2024-09-10T18:23:28.336+0000 7fbd8e002ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:23:28.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:28 smithi100 bash[18863]: debug 2024-09-10T18:23:28.396+0000 7fbd8e002ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:23:28.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:28 smithi100 bash[18863]: debug 2024-09-10T18:23:28.528+0000 7fbd8e002ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:23:29.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:29 smithi100 bash[18863]: debug 2024-09-10T18:23:29.148+0000 7fbd8e002ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:23:29.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:29 smithi100 bash[18863]: debug 2024-09-10T18:23:29.216+0000 7fbd8e002ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:23:29.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:29 smithi100 bash[18863]: debug 2024-09-10T18:23:29.292+0000 7fbd8e002ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:23:29.889 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:29 smithi100 bash[18863]: debug 2024-09-10T18:23:29.568+0000 7fbd8e002ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:23:29.890 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:29 smithi100 bash[18863]: debug 2024-09-10T18:23:29.700+0000 7fbd8e002ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:23:30.228 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:30 smithi100 bash[18863]: debug 2024-09-10T18:23:30.012+0000 7fbd8e002ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:23:30.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:30 smithi100 bash[18863]: debug 2024-09-10T18:23:30.224+0000 7fbd8e002ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:23:30.922 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:30 smithi100 bash[18863]: debug 2024-09-10T18:23:30.636+0000 7fbd8e002ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:23:30.922 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:30 smithi100 bash[18863]: debug 2024-09-10T18:23:30.700+0000 7fbd8e002ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:23:30.922 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:30 smithi100 bash[18863]: debug 2024-09-10T18:23:30.764+0000 7fbd8e002ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:23:30.923 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:30 smithi100 bash[18863]: debug 2024-09-10T18:23:30.856+0000 7fbd8e002ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:23:31.234 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:30 smithi100 bash[18863]: debug 2024-09-10T18:23:30.924+0000 7fbd8e002ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:23:31.234 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:31 smithi100 bash[18863]: debug 2024-09-10T18:23:31.000+0000 7fbd8e002ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:23:31.234 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:31 smithi100 bash[18863]: debug 2024-09-10T18:23:31.156+0000 7fbd8e002ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:23:31.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:31 smithi100 bash[18597]: audit 2024-09-10T18:23:31.248189+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? 172.21.15.100:0/4130505024' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:23:31.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:31 smithi100 bash[18863]: debug 2024-09-10T18:23:31.232+0000 7fbd8e002ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:23:31.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:31.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:23:31.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "505ebbde-6fa1-11ef-bcea-c7b262605968", 2024-09-10T18:23:31.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:31.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:23:31.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:23:31.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:31.911 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:23:17.238597+0000", 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e32329640 1 Processor -- start 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e32329640 1 -- start start 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e32329640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c0ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e32329640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0ffd90 con 0x7f1e2c0ff440 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e2bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c0ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e2bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c0ff850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46032/0 (socket says 172.21.15.100:46032) 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e2bfff640 1 -- 172.21.15.100:0/2114068831 learned_addr learned my addr 172.21.15.100:0/2114068831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.112+0000 7f1e2bfff640 1 -- 172.21.15.100:0/2114068831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0ffed0 con 0x7f1e2c0ff440 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e2bfff640 1 --2- 172.21.15.100:0/2114068831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c0ff850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009840 tx=0x7f1e1c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b16ed42f039191 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e2affd640 1 -- 172.21.15.100:0/2114068831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c02add0 con 0x7f1e2c0ff440 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e2affd640 1 -- 172.21.15.100:0/2114068831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f1e1c032890 con 0x7f1e2c0ff440 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e2affd640 1 -- 172.21.15.100:0/2114068831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c0317a0 con 0x7f1e2c0ff440 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 -- 172.21.15.100:0/2114068831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 msgr2=0x7f1e2c0ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 --2- 172.21.15.100:0/2114068831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c0ff850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009840 tx=0x7f1e1c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 -- 172.21.15.100:0/2114068831 shutdown_connections 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 --2- 172.21.15.100:0/2114068831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c0ff850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 -- 172.21.15.100:0/2114068831 >> 172.21.15.100:0/2114068831 conn(0x7f1e2c0fabc0 msgr2=0x7f1e2c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 -- 172.21.15.100:0/2114068831 shutdown_connections 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 -- 172.21.15.100:0/2114068831 wait complete. 2024-09-10T18:23:31.913 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 Processor -- start 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.116+0000 7f1e32329640 1 -- start start 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e32329640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e32329640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c111fd0 con 0x7f1e2c0ff440 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e2bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e2bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46040/0 (socket says 172.21.15.100:46040) 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e2bfff640 1 -- 172.21.15.100:0/4130505024 learned_addr learned my addr 172.21.15.100:0/4130505024 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e2bfff640 1 -- 172.21.15.100:0/4130505024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c009450 con 0x7f1e2c0ff440 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e2bfff640 1 --2- 172.21.15.100:0/4130505024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c074b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0314e0 tx=0x7f1e1c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e28ff9640 1 -- 172.21.15.100:0/4130505024 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c03c400 con 0x7f1e2c0ff440 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e28ff9640 1 -- 172.21.15.100:0/4130505024 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f1e1c03c9b0 con 0x7f1e2c0ff440 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c112110 con 0x7f1e2c0ff440 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e28ff9640 1 -- 172.21.15.100:0/4130505024 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e1c03a530 con 0x7f1e2c0ff440 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c112520 con 0x7f1e2c0ff440 2024-09-10T18:23:31.914 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e28ff9640 1 -- 172.21.15.100:0/4130505024 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1e1c03a920 con 0x7f1e2c0ff440 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e28ff9640 1 -- 172.21.15.100:0/4130505024 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1e1c046c00 con 0x7f1e2c0ff440 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.120+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c05ed70 con 0x7f1e2c0ff440 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.124+0000 7f1e28ff9640 1 -- 172.21.15.100:0/4130505024 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f1e1c046e60 con 0x7f1e2c0ff440 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.244+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1e2c101260 con 0x7f1e2c0ff440 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.244+0000 7f1e28ff9640 1 -- 172.21.15.100:0/4130505024 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f1e1c055090 con 0x7f1e2c0ff440 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.248+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 msgr2=0x7f1e2c074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.248+0000 7f1e32329640 1 --2- 172.21.15.100:0/4130505024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c074b00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0314e0 tx=0x7f1e1c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.248+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 shutdown_connections 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.248+0000 7f1e32329640 1 --2- 172.21.15.100:0/4130505024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e2c0ff440 0x7f1e2c074b00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.248+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 >> 172.21.15.100:0/4130505024 conn(0x7f1e2c0fabc0 msgr2=0x7f1e2c0fb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.248+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 shutdown_connections 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:31.248+0000 7f1e32329640 1 -- 172.21.15.100:0/4130505024 wait complete. 2024-09-10T18:23:31.915 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (2/15)... 2024-09-10T18:23:32.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18863]: debug 2024-09-10T18:23:32.156+0000 7fbd8e002ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:23:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: cluster 2024-09-10T18:23:32.160581+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-09-10T18:23:32.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: cluster 2024-09-10T18:23:32.167253+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.0068905s) 2024-09-10T18:23:32.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.167988+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:23:32.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.168090+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:23:32.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.168189+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:23:32.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.168823+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:23:32.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.170313+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:23:32.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: cluster 2024-09-10T18:23:32.179172+0000 mon.a (mon.0) 16 : cluster [INF] Manager daemon y is now available 2024-09-10T18:23:32.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.196870+0000 mon.a (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:23:32.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.200358+0000 mon.a (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:23:32.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.206440+0000 mon.a (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' 2024-09-10T18:23:32.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.210033+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' 2024-09-10T18:23:32.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:32 smithi100 bash[18597]: audit 2024-09-10T18:23:32.213738+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.100:0/4286724864' entity='mgr.y' 2024-09-10T18:23:34.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:34 smithi100 bash[18597]: cluster 2024-09-10T18:23:33.174046+0000 mon.a (mon.0) 22 : cluster [DBG] mgrmap e3: y(active, since 1.01368s) 2024-09-10T18:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "505ebbde-6fa1-11ef-bcea-c7b262605968", 2024-09-10T18:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:23:36.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:23:36.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:36.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:23:36.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:23:36.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:23:36.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:23:36.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:23:36.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:23:36.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:23:36.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:23:17.238597+0000", 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65fd9dc640 1 Processor -- start 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65fd9dc640 1 -- start start 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65fd9dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8077f70 0x7f65f8078380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65fd9dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f80788c0 con 0x7f65f8077f70 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65f6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8077f70 0x7f65f8078380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65f6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8077f70 0x7f65f8078380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46092/0 (socket says 172.21.15.100:46092) 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65f6ffd640 1 -- 172.21.15.100:0/130648328 learned_addr learned my addr 172.21.15.100:0/130648328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:36.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65f6ffd640 1 -- 172.21.15.100:0/130648328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8078a00 con 0x7f65f8077f70 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65f6ffd640 1 --2- 172.21.15.100:0/130648328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8077f70 0x7f65f8078380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009840 tx=0x7f65e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bd82ec04ca56ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65f5ffb640 1 -- 172.21.15.100:0/130648328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e002add0 con 0x7f65f8077f70 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.140+0000 7f65f5ffb640 1 -- 172.21.15.100:0/130648328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f65e0032890 con 0x7f65f8077f70 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65f5ffb640 1 -- 172.21.15.100:0/130648328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e00317a0 con 0x7f65f8077f70 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- 172.21.15.100:0/130648328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8077f70 msgr2=0x7f65f8078380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 --2- 172.21.15.100:0/130648328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8077f70 0x7f65f8078380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009840 tx=0x7f65e00091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- 172.21.15.100:0/130648328 shutdown_connections 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 --2- 172.21.15.100:0/130648328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f8077f70 0x7f65f8078380 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- 172.21.15.100:0/130648328 >> 172.21.15.100:0/130648328 conn(0x7f65f8075de0 msgr2=0x7f65f8076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- 172.21.15.100:0/130648328 shutdown_connections 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- 172.21.15.100:0/130648328 wait complete. 2024-09-10T18:23:36.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 Processor -- start 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- start start 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f81159a0 0x7f65f8115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f81162f0 con 0x7f65f81159a0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65f6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f81159a0 0x7f65f8115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65f6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f81159a0 0x7f65f8115db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46106/0 (socket says 172.21.15.100:46106) 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65f6ffd640 1 -- 172.21.15.100:0/894704311 learned_addr learned my addr 172.21.15.100:0/894704311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65f6ffd640 1 -- 172.21.15.100:0/894704311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0009450 con 0x7f65f81159a0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65f6ffd640 1 --2- 172.21.15.100:0/894704311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f81159a0 0x7f65f8115db0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f65e00313e0 tx=0x7f65e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e0032cd0 con 0x7f65f81159a0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f65e0031ec0 con 0x7f65f81159a0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65e003b420 con 0x7f65f81159a0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.144+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f81164f0 con 0x7f65f81159a0 2024-09-10T18:23:36.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.148+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f8116ed0 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.148+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f805ed70 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.148+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 3) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f65e0039070 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.148+0000 7f65ef7fe640 1 --2- 172.21.15.100:0/894704311 >> v2:172.21.15.100:6800/1728522861 conn(0x7f65e40392c0 0x7f65e403b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.148+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f65e006ca40 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.148+0000 7f65f67fc640 1 --2- 172.21.15.100:0/894704311 >> v2:172.21.15.100:6800/1728522861 conn(0x7f65e40392c0 0x7f65e403b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.148+0000 7f65f67fc640 1 --2- 172.21.15.100:0/894704311 >> v2:172.21.15.100:6800/1728522861 conn(0x7f65e40392c0 0x7f65e403b770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f65e800ad90 tx=0x7f65e8009410 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.156+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f65e003c590 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.168+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 4) v1 ==== 45446+0+0 (secure 0 0 0) 0x7f65e003d970 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.600+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f65f804b720 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65ef7fe640 1 -- 172.21.15.100:0/894704311 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f65e0037020 con 0x7f65f81159a0 2024-09-10T18:23:36.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 >> v2:172.21.15.100:6800/1728522861 conn(0x7f65e40392c0 msgr2=0x7f65e403b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 --2- 172.21.15.100:0/894704311 >> v2:172.21.15.100:6800/1728522861 conn(0x7f65e40392c0 0x7f65e403b770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f65e800ad90 tx=0x7f65e8009410 comp rx=0 tx=0).stop 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f81159a0 msgr2=0x7f65f8115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 --2- 172.21.15.100:0/894704311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f81159a0 0x7f65f8115db0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f65e00313e0 tx=0x7f65e00091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 shutdown_connections 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 --2- 172.21.15.100:0/894704311 >> v2:172.21.15.100:6800/1728522861 conn(0x7f65e40392c0 0x7f65e403b770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 --2- 172.21.15.100:0/894704311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f81159a0 0x7f65f8115db0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 >> 172.21.15.100:0/894704311 conn(0x7f65f8075de0 msgr2=0x7f65f8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 shutdown_connections 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:35.604+0000 7f65fd9dc640 1 -- 172.21.15.100:0/894704311 wait complete. 2024-09-10T18:23:36.159 INFO:teuthology.orchestra.run.smithi100.stdout:mgr is available 2024-09-10T18:23:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:36 smithi100 bash[18597]: cluster 2024-09-10T18:23:35.172378+0000 mon.a (mon.0) 23 : cluster [DBG] mgrmap e4: y(active, since 3s) 2024-09-10T18:23:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:36 smithi100 bash[18597]: audit 2024-09-10T18:23:35.605777+0000 mon.a (mon.0) 24 : audit [DBG] from='client.? 172.21.15.100:0/894704311' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:23:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:37 smithi100 bash[18597]: audit 2024-09-10T18:23:37.636822+0000 mon.a (mon.0) 25 : audit [INF] from='client.? 172.21.15.100:0/3198830323' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:23:38.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:38.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:23:38.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:38.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:23:38.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:23:38.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:23:38.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.296+0000 7f8c063b2640 1 Processor -- start 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.296+0000 7f8c063b2640 1 -- start start 2024-09-10T18:23:38.222 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.296+0000 7f8c063b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.296+0000 7f8c063b2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00103160 con 0x7f8c00102810 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.296+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.296+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46118/0 (socket says 172.21.15.100:46118) 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.296+0000 7f8bfffff640 1 -- 172.21.15.100:0/528483451 learned_addr learned my addr 172.21.15.100:0/528483451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8bfffff640 1 -- 172.21.15.100:0/528483451 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c001032a0 con 0x7f8c00102810 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8bfffff640 1 --2- 172.21.15.100:0/528483451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00102c20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009840 tx=0x7f8bf00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05dd6c3a00974d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8bfeffd640 1 -- 172.21.15.100:0/528483451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf002add0 con 0x7f8c00102810 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8bfeffd640 1 -- 172.21.15.100:0/528483451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f8bf0032890 con 0x7f8c00102810 2024-09-10T18:23:38.223 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8bfeffd640 1 -- 172.21.15.100:0/528483451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf00317a0 con 0x7f8c00102810 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 -- 172.21.15.100:0/528483451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 msgr2=0x7f8c00102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 --2- 172.21.15.100:0/528483451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00102c20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009840 tx=0x7f8bf00091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 -- 172.21.15.100:0/528483451 shutdown_connections 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 --2- 172.21.15.100:0/528483451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00102c20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 -- 172.21.15.100:0/528483451 >> 172.21.15.100:0/528483451 conn(0x7f8c00075de0 msgr2=0x7f8c00076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 -- 172.21.15.100:0/528483451 shutdown_connections 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 -- 172.21.15.100:0/528483451 wait complete. 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 Processor -- start 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 -- start start 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.300+0000 7f8c063b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00111440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8c063b2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00111980 con 0x7f8c00102810 2024-09-10T18:23:38.224 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00111440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00111440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46132/0 (socket says 172.21.15.100:46132) 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfffff640 1 -- 172.21.15.100:0/3198830323 learned_addr learned my addr 172.21.15.100:0/3198830323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfffff640 1 -- 172.21.15.100:0/3198830323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0009450 con 0x7f8c00102810 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfffff640 1 --2- 172.21.15.100:0/3198830323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00111440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00313e0 tx=0x7f8bf00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfcff9640 1 -- 172.21.15.100:0/3198830323 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0031e70 con 0x7f8c00102810 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c00111b80 con 0x7f8c00102810 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfcff9640 1 -- 172.21.15.100:0/3198830323 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f8bf003c850 con 0x7f8c00102810 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfcff9640 1 -- 172.21.15.100:0/3198830323 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bf003a540 con 0x7f8c00102810 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c00111ff0 con 0x7f8c00102810 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.304+0000 7f8bfcff9640 1 -- 172.21.15.100:0/3198830323 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 45446+0+0 (secure 0 0 0) 0x7f8bf003cc10 con 0x7f8c00102810 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.308+0000 7f8bfcff9640 1 --2- 172.21.15.100:0/3198830323 >> v2:172.21.15.100:6800/1728522861 conn(0x7f8bec0393e0 0x7f8bec03b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:38.225 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.308+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0004b720 con 0x7f8c00102810 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.308+0000 7f8bff7fe640 1 --2- 172.21.15.100:0/3198830323 >> v2:172.21.15.100:6800/1728522861 conn(0x7f8bec0393e0 0x7f8bec03b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.308+0000 7f8bfcff9640 1 -- 172.21.15.100:0/3198830323 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8bf006c930 con 0x7f8c00102810 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.308+0000 7f8bff7fe640 1 --2- 172.21.15.100:0/3198830323 >> v2:172.21.15.100:6800/1728522861 conn(0x7f8bec0393e0 0x7f8bec03b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.312+0000 7f8bfcff9640 1 -- 172.21.15.100:0/3198830323 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8bf003c400 con 0x7f8c00102810 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.632+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8c00111ff0 con 0x7f8c00102810 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.636+0000 7f8bfcff9640 1 -- 172.21.15.100:0/3198830323 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+318 (secure 0 0 0) 0x7f8bf002f2a0 con 0x7f8c00102810 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 >> v2:172.21.15.100:6800/1728522861 conn(0x7f8bec0393e0 msgr2=0x7f8bec03b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 --2- 172.21.15.100:0/3198830323 >> v2:172.21.15.100:6800/1728522861 conn(0x7f8bec0393e0 0x7f8bec03b890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8be8006fd0 tx=0x7f8be8006db0 comp rx=0 tx=0).stop 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 msgr2=0x7f8c00111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 --2- 172.21.15.100:0/3198830323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00111440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00313e0 tx=0x7f8bf00091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 shutdown_connections 2024-09-10T18:23:38.226 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 --2- 172.21.15.100:0/3198830323 >> v2:172.21.15.100:6800/1728522861 conn(0x7f8bec0393e0 0x7f8bec03b890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:38.227 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 --2- 172.21.15.100:0/3198830323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c00102810 0x7f8c00111440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:38.227 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 >> 172.21.15.100:0/3198830323 conn(0x7f8c00075de0 msgr2=0x7f8c00104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:38.227 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 shutdown_connections 2024-09-10T18:23:38.227 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:37.640+0000 7f8c063b2640 1 -- 172.21.15.100:0/3198830323 wait complete. 2024-09-10T18:23:38.227 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling cephadm module... 2024-09-10T18:23:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:39 smithi100 bash[18597]: audit 2024-09-10T18:23:39.753351+0000 mon.a (mon.0) 26 : audit [INF] from='client.? 172.21.15.100:0/1747496763' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:23:40.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:39 smithi100 bash[18863]: ignoring --setuser ceph since I am not root 2024-09-10T18:23:40.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:39 smithi100 bash[18863]: ignoring --setgroup ceph since I am not root 2024-09-10T18:23:40.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:39 smithi100 bash[18863]: debug 2024-09-10T18:23:39.968+0000 7fa4e83d8ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad8d7d9640 1 Processor -- start 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad8d7d9640 1 -- start start 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad8d7d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad88100a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad8d7d9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88100f50 con 0x7fad88100600 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad86ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad88100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad86ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad88100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46136/0 (socket says 172.21.15.100:46136) 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad86ffd640 1 -- 172.21.15.100:0/440784188 learned_addr learned my addr 172.21.15.100:0/440784188 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad86ffd640 1 -- 172.21.15.100:0/440784188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88101090 con 0x7fad88100600 2024-09-10T18:23:40.408 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.324+0000 7fad86ffd640 1 --2- 172.21.15.100:0/440784188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad88100a10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009840 tx=0x7fad7c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f6128d30a01b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad85ffb640 1 -- 172.21.15.100:0/440784188 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad7c02add0 con 0x7fad88100600 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad85ffb640 1 -- 172.21.15.100:0/440784188 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fad7c032890 con 0x7fad88100600 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad85ffb640 1 -- 172.21.15.100:0/440784188 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad7c0317a0 con 0x7fad88100600 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 -- 172.21.15.100:0/440784188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 msgr2=0x7fad88100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 --2- 172.21.15.100:0/440784188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad88100a10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009840 tx=0x7fad7c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 -- 172.21.15.100:0/440784188 shutdown_connections 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 --2- 172.21.15.100:0/440784188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad88100a10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 -- 172.21.15.100:0/440784188 >> 172.21.15.100:0/440784188 conn(0x7fad880fbe10 msgr2=0x7fad880fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 -- 172.21.15.100:0/440784188 shutdown_connections 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 -- 172.21.15.100:0/440784188 wait complete. 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 Processor -- start 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 -- start start 2024-09-10T18:23:40.409 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad8810f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad8d7d9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8810f7c0 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad86ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad8810f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad86ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad8810f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46146/0 (socket says 172.21.15.100:46146) 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad86ffd640 1 -- 172.21.15.100:0/1747496763 learned_addr learned my addr 172.21.15.100:0/1747496763 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad86ffd640 1 -- 172.21.15.100:0/1747496763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c009450 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.328+0000 7fad86ffd640 1 --2- 172.21.15.100:0/1747496763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad8810f280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0313e0 tx=0x7fad7c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad7c031e70 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fad7c03c850 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8810f9c0 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad7c03a540 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8810fe30 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 45446+0+0 (secure 0 0 0) 0x7fad7c03a7c0 con 0x7fad88100600 2024-09-10T18:23:40.410 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad6f7fe640 1 --2- 172.21.15.100:0/1747496763 >> v2:172.21.15.100:6800/1728522861 conn(0x7fad700393e0 0x7fad7003b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fad7c06d730 con 0x7fad88100600 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad867fc640 1 --2- 172.21.15.100:0/1747496763 >> v2:172.21.15.100:6800/1728522861 conn(0x7fad700393e0 0x7fad7003b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8805ed70 con 0x7fad88100600 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.332+0000 7fad867fc640 1 --2- 172.21.15.100:0/1747496763 >> v2:172.21.15.100:6800/1728522861 conn(0x7fad700393e0 0x7fad7003b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad78006fd0 tx=0x7fad78006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.340+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fad7c037030 con 0x7fad88100600 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.748+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fad88102510 con 0x7fad88100600 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.804+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 5) v1 ==== 45457+0+0 (secure 0 0 0) 0x7fad7c04cd10 con 0x7fad88100600 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.808+0000 7fad6f7fe640 1 -- 172.21.15.100:0/1747496763 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fad7c0396d0 con 0x7fad88100600 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 >> v2:172.21.15.100:6800/1728522861 conn(0x7fad700393e0 msgr2=0x7fad7003b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 --2- 172.21.15.100:0/1747496763 >> v2:172.21.15.100:6800/1728522861 conn(0x7fad700393e0 0x7fad7003b890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad78006fd0 tx=0x7fad78006db0 comp rx=0 tx=0).stop 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 msgr2=0x7fad8810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 --2- 172.21.15.100:0/1747496763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad8810f280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0313e0 tx=0x7fad7c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:40.411 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 shutdown_connections 2024-09-10T18:23:40.412 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 --2- 172.21.15.100:0/1747496763 >> v2:172.21.15.100:6800/1728522861 conn(0x7fad700393e0 0x7fad7003b890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:40.412 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 --2- 172.21.15.100:0/1747496763 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad88100600 0x7fad8810f280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:40.412 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 >> 172.21.15.100:0/1747496763 conn(0x7fad880fbe10 msgr2=0x7fad880fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:40.412 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 shutdown_connections 2024-09-10T18:23:40.412 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:39.812+0000 7fad8d7d9640 1 -- 172.21.15.100:0/1747496763 wait complete. 2024-09-10T18:23:40.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:40 smithi100 bash[18863]: debug 2024-09-10T18:23:40.728+0000 7fa4e83d8ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:23:41.264 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:40 smithi100 bash[18597]: audit 2024-09-10T18:23:39.810624+0000 mon.a (mon.0) 27 : audit [INF] from='client.? 172.21.15.100:0/1747496763' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:23:41.264 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:40 smithi100 bash[18597]: cluster 2024-09-10T18:23:39.810796+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e5: y(active, since 7s) 2024-09-10T18:23:41.264 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:40 smithi100 bash[18863]: debug 2024-09-10T18:23:40.792+0000 7fa4e83d8ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:23:41.264 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:40 smithi100 bash[18863]: debug 2024-09-10T18:23:40.860+0000 7fa4e83d8ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:23:41.264 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:41 smithi100 bash[18863]: debug 2024-09-10T18:23:41.012+0000 7fa4e83d8ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:23:41.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:41 smithi100 bash[18863]: debug 2024-09-10T18:23:41.260+0000 7fa4e83d8ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:23:41.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:41 smithi100 bash[18863]: debug 2024-09-10T18:23:41.324+0000 7fa4e83d8ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:23:41.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:41 smithi100 bash[18863]: debug 2024-09-10T18:23:41.472+0000 7fa4e83d8ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:23:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:42 smithi100 bash[18597]: audit 2024-09-10T18:23:41.997356+0000 mon.a (mon.0) 29 : audit [DBG] from='client.? 172.21.15.100:0/1855718752' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:23:42.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:42 smithi100 bash[18863]: debug 2024-09-10T18:23:42.120+0000 7fa4e83d8ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:23:42.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:42 smithi100 bash[18863]: debug 2024-09-10T18:23:42.192+0000 7fa4e83d8ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:23:42.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:42 smithi100 bash[18863]: debug 2024-09-10T18:23:42.268+0000 7fa4e83d8ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:23:42.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:23:42.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T18:23:42.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:23:42.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:23:42.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:23:42.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:23:42.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7edc2b5640 1 Processor -- start 2024-09-10T18:23:42.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7edc2b5640 1 -- start start 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7edc2b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed4100a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7edc2b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed4100f90 con 0x7f7ed4100640 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7eda051640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed4100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7eda051640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed4100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54048/0 (socket says 172.21.15.100:54048) 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7eda051640 1 -- 172.21.15.100:0/2398295086 learned_addr learned my addr 172.21.15.100:0/2398295086 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7eda051640 1 -- 172.21.15.100:0/2398295086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed41010d0 con 0x7f7ed4100640 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.568+0000 7f7eda051640 1 --2- 172.21.15.100:0/2398295086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed4100a50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009840 tx=0x7f7ed00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5902f9c440ad2234 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7ed904f640 1 -- 172.21.15.100:0/2398295086 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed002add0 con 0x7f7ed4100640 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7ed904f640 1 -- 172.21.15.100:0/2398295086 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f7ed0032890 con 0x7f7ed4100640 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7ed904f640 1 -- 172.21.15.100:0/2398295086 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed00317a0 con 0x7f7ed4100640 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- 172.21.15.100:0/2398295086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 msgr2=0x7f7ed4100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:42.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 --2- 172.21.15.100:0/2398295086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed4100a50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009840 tx=0x7f7ed00091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- 172.21.15.100:0/2398295086 shutdown_connections 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 --2- 172.21.15.100:0/2398295086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed4100a50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- 172.21.15.100:0/2398295086 >> 172.21.15.100:0/2398295086 conn(0x7f7ed40fbe10 msgr2=0x7f7ed40fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- 172.21.15.100:0/2398295086 shutdown_connections 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- 172.21.15.100:0/2398295086 wait complete. 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 Processor -- start 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- start start 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed40774b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40779f0 con 0x7f7ed4100640 2024-09-10T18:23:42.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7eda051640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed40774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:42.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7eda051640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed40774b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54054/0 (socket says 172.21.15.100:54054) 2024-09-10T18:23:42.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7eda051640 1 -- 172.21.15.100:0/1855718752 learned_addr learned my addr 172.21.15.100:0/1855718752 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:42.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7eda051640 1 -- 172.21.15.100:0/1855718752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0009450 con 0x7f7ed4100640 2024-09-10T18:23:42.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7eda051640 1 --2- 172.21.15.100:0/1855718752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed40774b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0000c00 tx=0x7f7ed00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7ecaffd640 1 -- 172.21.15.100:0/1855718752 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed0032d20 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7ecaffd640 1 -- 172.21.15.100:0/1855718752 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f7ed003c850 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed4075a80 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.572+0000 7f7ecaffd640 1 -- 172.21.15.100:0/1855718752 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed003a520 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.576+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed4075ef0 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.576+0000 7f7ecaffd640 1 -- 172.21.15.100:0/1855718752 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f7ed003a680 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.576+0000 7f7ecaffd640 1 --2- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 0x7f7ec003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.576+0000 7f7ecaffd640 1 -- 172.21.15.100:0/1855718752 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7ed006cd30 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.576+0000 7f7ed9850640 1 -- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 msgr2=0x7f7ec003b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1728522861 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.576+0000 7f7ed9850640 1 --2- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 0x7f7ec003b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.576+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed404b720 con 0x7f7ed4100640 2024-09-10T18:23:42.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.584+0000 7f7ecaffd640 1 -- 172.21.15.100:0/1855718752 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f7ed0037070 con 0x7f7ed4100640 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.780+0000 7f7ed9850640 1 -- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 msgr2=0x7f7ec003b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1728522861 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.780+0000 7f7ed9850640 1 --2- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 0x7f7ec003b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.992+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7ed404b720 con 0x7f7ed4100640 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7ecaffd640 1 -- 172.21.15.100:0/1855718752 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f7ed003acb0 con 0x7f7ed4100640 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 msgr2=0x7f7ec003b920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 --2- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 0x7f7ec003b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 msgr2=0x7f7ed40774b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 --2- 172.21.15.100:0/1855718752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed40774b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0000c00 tx=0x7f7ed00091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 shutdown_connections 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 --2- 172.21.15.100:0/1855718752 >> v2:172.21.15.100:6800/1728522861 conn(0x7f7ec0039470 0x7f7ec003b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 --2- 172.21.15.100:0/1855718752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ed4100640 0x7f7ed40774b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 >> 172.21.15.100:0/1855718752 conn(0x7f7ed40fbe10 msgr2=0x7f7ed40fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 shutdown_connections 2024-09-10T18:23:42.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:41.996+0000 7f7edc2b5640 1 -- 172.21.15.100:0/1855718752 wait complete. 2024-09-10T18:23:42.631 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-09-10T18:23:42.631 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 5... 2024-09-10T18:23:43.003 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:42 smithi100 bash[18863]: debug 2024-09-10T18:23:42.548+0000 7fa4e83d8ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:23:43.003 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:42 smithi100 bash[18863]: debug 2024-09-10T18:23:42.687+0000 7fa4e83d8ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:23:43.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:43 smithi100 bash[18863]: debug 2024-09-10T18:23:42.999+0000 7fa4e83d8ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:23:43.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:43 smithi100 bash[18863]: debug 2024-09-10T18:23:43.215+0000 7fa4e83d8ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:23:43.941 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:43 smithi100 bash[18863]: debug 2024-09-10T18:23:43.627+0000 7fa4e83d8ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:23:43.941 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:43 smithi100 bash[18863]: debug 2024-09-10T18:23:43.691+0000 7fa4e83d8ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:23:43.941 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:43 smithi100 bash[18863]: debug 2024-09-10T18:23:43.763+0000 7fa4e83d8ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:23:43.941 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:43 smithi100 bash[18863]: debug 2024-09-10T18:23:43.867+0000 7fa4e83d8ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:23:43.942 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:43 smithi100 bash[18863]: debug 2024-09-10T18:23:43.935+0000 7fa4e83d8ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:23:44.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:44 smithi100 bash[18863]: debug 2024-09-10T18:23:44.011+0000 7fa4e83d8ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:23:44.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:44 smithi100 bash[18863]: debug 2024-09-10T18:23:44.155+0000 7fa4e83d8ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:23:44.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:44 smithi100 bash[18863]: debug 2024-09-10T18:23:44.231+0000 7fa4e83d8ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:23:45.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18863]: debug 2024-09-10T18:23:45.151+0000 7fa4e83d8ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:23:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: cluster 2024-09-10T18:23:45.175892+0000 mon.a (mon.0) 30 : cluster [INF] Active manager daemon y restarted 2024-09-10T18:23:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: cluster 2024-09-10T18:23:45.176340+0000 mon.a (mon.0) 31 : cluster [INF] Activating manager daemon y 2024-09-10T18:23:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: cluster 2024-09-10T18:23:45.180853+0000 mon.a (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:23:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: cluster 2024-09-10T18:23:45.181060+0000 mon.a (mon.0) 33 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00488979s) 2024-09-10T18:23:45.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: audit 2024-09-10T18:23:45.184192+0000 mon.a (mon.0) 34 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:23:45.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: audit 2024-09-10T18:23:45.186415+0000 mon.a (mon.0) 35 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:23:45.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: audit 2024-09-10T18:23:45.187514+0000 mon.a (mon.0) 36 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:23:45.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: audit 2024-09-10T18:23:45.187773+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:23:45.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: audit 2024-09-10T18:23:45.187944+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:23:45.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: cluster 2024-09-10T18:23:45.198381+0000 mon.a (mon.0) 39 : cluster [INF] Manager daemon y is now available 2024-09-10T18:23:45.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:45 smithi100 bash[18597]: audit 2024-09-10T18:23:45.224781+0000 mon.a (mon.0) 40 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:23:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:46 smithi100 bash[18597]: cephadm 2024-09-10T18:23:45.216714+0000 mgr.y (mgr.14118) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-09-10T18:23:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:46 smithi100 bash[18597]: audit 2024-09-10T18:23:45.239583+0000 mon.a (mon.0) 41 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:23:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:46 smithi100 bash[18597]: audit 2024-09-10T18:23:45.245184+0000 mon.a (mon.0) 42 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:46 smithi100 bash[18597]: audit 2024-09-10T18:23:45.251229+0000 mon.a (mon.0) 43 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:46 smithi100 bash[18597]: audit 2024-09-10T18:23:45.270527+0000 mon.a (mon.0) 44 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:23:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:46 smithi100 bash[18597]: audit 2024-09-10T18:23:45.278835+0000 mon.a (mon.0) 45 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:23:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:46 smithi100 bash[18597]: cluster 2024-09-10T18:23:46.188221+0000 mon.a (mon.0) 46 : cluster [DBG] mgrmap e7: y(active, since 1.01205s) 2024-09-10T18:23:46.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:23:46.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T18:23:46.859 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:23:46.859 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:23:46.860 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87fe3a9640 1 Processor -- start 2024-09-10T18:23:46.860 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87fe3a9640 1 -- start start 2024-09-10T18:23:46.860 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87fe3a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f80ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:46.860 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87fe3a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80ffd90 con 0x7f87f80ff440 2024-09-10T18:23:46.860 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f80ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:46.860 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f80ff850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54066/0 (socket says 172.21.15.100:54066) 2024-09-10T18:23:46.860 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f7fff640 1 -- 172.21.15.100:0/2667613118 learned_addr learned my addr 172.21.15.100:0/2667613118 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:46.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f7fff640 1 -- 172.21.15.100:0/2667613118 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80ffed0 con 0x7f87f80ff440 2024-09-10T18:23:46.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f7fff640 1 --2- 172.21.15.100:0/2667613118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f80ff850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f87e800baf0 tx=0x7f87e800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5165cc961f1f3f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:46.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f6ffd640 1 -- 172.21.15.100:0/2667613118 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87e8032640 con 0x7f87f80ff440 2024-09-10T18:23:46.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f6ffd640 1 -- 172.21.15.100:0/2667613118 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f87e8032bf0 con 0x7f87f80ff440 2024-09-10T18:23:46.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.907+0000 7f87f6ffd640 1 -- 172.21.15.100:0/2667613118 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87e8031b50 con 0x7f87f80ff440 2024-09-10T18:23:46.863 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- 172.21.15.100:0/2667613118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 msgr2=0x7f87f80ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:46.863 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 --2- 172.21.15.100:0/2667613118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f80ff850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f87e800baf0 tx=0x7f87e800b460 comp rx=0 tx=0).stop 2024-09-10T18:23:46.863 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- 172.21.15.100:0/2667613118 shutdown_connections 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 --2- 172.21.15.100:0/2667613118 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f80ff850 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- 172.21.15.100:0/2667613118 >> 172.21.15.100:0/2667613118 conn(0x7f87f80fabc0 msgr2=0x7f87f80fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- 172.21.15.100:0/2667613118 shutdown_connections 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- 172.21.15.100:0/2667613118 wait complete. 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 Processor -- start 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- start start 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f81158c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8115e00 con 0x7f87f80ff440 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f81158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f81158c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54080/0 (socket says 172.21.15.100:54080) 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f7fff640 1 -- 172.21.15.100:0/919986368 learned_addr learned my addr 172.21.15.100:0/919986368 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f7fff640 1 -- 172.21.15.100:0/919986368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e800b700 con 0x7f87f80ff440 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f7fff640 1 --2- 172.21.15.100:0/919986368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f81158c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87e8003080 tx=0x7f87e803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:46.864 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87e8031b40 con 0x7f87f80ff440 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f87e8031ca0 con 0x7f87f80ff440 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87e80453f0 con 0x7f87f80ff440 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f8116000 con 0x7f87f80ff440 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.911+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f8116470 con 0x7f87f80ff440 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.915+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f87e8045660 con 0x7f87f80ff440 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.915+0000 7f87f4ff9640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 0x7f87e003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.915+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f87e806f8e0 con 0x7f87f80ff440 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.915+0000 7f87f77fe640 1 -- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 msgr2=0x7f87e003b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1728522861 2024-09-10T18:23:46.865 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.915+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 --> v2:172.21.15.100:6800/1728522861 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87f805ed70 con 0x7f87e0039470 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:43.915+0000 7f87f77fe640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 0x7f87e003b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:44.119+0000 7f87f77fe640 1 -- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 msgr2=0x7f87e003b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1728522861 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:44.119+0000 7f87f77fe640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 0x7f87e003b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:44.523+0000 7f87f77fe640 1 -- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 msgr2=0x7f87e003b920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1728522861 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:44.523+0000 7f87f77fe640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 0x7f87e003b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:45.175+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 6) v1 ==== 45259+0+0 (secure 0 0 0) 0x7f87e806ead0 con 0x7f87f80ff440 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:45.175+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 msgr2=0x7f87e003b920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:45.175+0000 7f87f4ff9640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 0x7f87e003b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.179+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 7) v1 ==== 45351+0+0 (secure 0 0 0) 0x7f87e804cd10 con 0x7f87f80ff440 2024-09-10T18:23:46.868 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.179+0000 7f87f4ff9640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/2893280940 conn(0x7f87e003cbe0 0x7f87e003efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.179+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 --> v2:172.21.15.100:6800/2893280940 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87f805ed70 con 0x7f87e003cbe0 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.179+0000 7f87f77fe640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/2893280940 conn(0x7f87e003cbe0 0x7f87e003efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.183+0000 7f87f77fe640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/2893280940 conn(0x7f87e003cbe0 0x7f87e003efc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f87ec003940 tx=0x7f87ec006ec0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.183+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f87f805ed70 con 0x7f87e003cbe0 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 --> v2:172.21.15.100:6800/2893280940 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f87f804b720 con 0x7f87e003cbe0 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87f4ff9640 1 -- 172.21.15.100:0/919986368 <== mgr.14118 v2:172.21.15.100:6800/2893280940 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f87f804b720 con 0x7f87e003cbe0 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/2893280940 conn(0x7f87e003cbe0 msgr2=0x7f87e003efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/2893280940 conn(0x7f87e003cbe0 0x7f87e003efc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f87ec003940 tx=0x7f87ec006ec0 comp rx=0 tx=0).stop 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 msgr2=0x7f87f81158c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 --2- 172.21.15.100:0/919986368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f81158c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f87e8003080 tx=0x7f87e803d040 comp rx=0 tx=0).stop 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 shutdown_connections 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/2893280940 conn(0x7f87e003cbe0 0x7f87e003efc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 --2- 172.21.15.100:0/919986368 >> v2:172.21.15.100:6800/1728522861 conn(0x7f87e0039470 0x7f87e003b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:46.869 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 --2- 172.21.15.100:0/919986368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff440 0x7f87f81158c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:46.870 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 >> 172.21.15.100:0/919986368 conn(0x7f87f80fabc0 msgr2=0x7f87f8069140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:46.870 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 shutdown_connections 2024-09-10T18:23:46.870 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:46.195+0000 7f87fe3a9640 1 -- 172.21.15.100:0/919986368 wait complete. 2024-09-10T18:23:46.870 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 5 is available 2024-09-10T18:23:46.870 INFO:teuthology.orchestra.run.smithi100.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:23:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18597]: audit 2024-09-10T18:23:46.185917+0000 mgr.y (mgr.14118) 2 : audit [DBG] from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:23:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18597]: audit 2024-09-10T18:23:46.198994+0000 mgr.y (mgr.14118) 3 : audit [DBG] from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:23:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18597]: audit 2024-09-10T18:23:47.219927+0000 mon.a (mon.0) 47 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:23:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18597]: audit 2024-09-10T18:23:47.221679+0000 mon.a (mon.0) 48 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: debug 2024-09-10T18:23:47.215+0000 7fa49a7ac640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:23:47] ENGINE Error in HTTPServer.serve 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: Traceback (most recent call last): 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: self._connections.run(self.expiration_interval) 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: self._run(expiration_interval) 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:23:47.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: s = self.context.wrap_socket( 2024-09-10T18:23:47.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:23:47.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: return self.sslsocket_class._create( 2024-09-10T18:23:47.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:23:47.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: self.do_handshake() 2024-09-10T18:23:47.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:23:47.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: self._sslobj.do_handshake() 2024-09-10T18:23:47.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:47 smithi100 bash[18863]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:23:48.531 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: cephadm 2024-09-10T18:23:47.085925+0000 mgr.y (mgr.14118) 4 : cephadm [INF] [10/Sep/2024:18:23:47] ENGINE Bus STARTING 2024-09-10T18:23:48.531 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: cephadm 2024-09-10T18:23:47.213239+0000 mgr.y (mgr.14118) 5 : cephadm [INF] [10/Sep/2024:18:23:47] ENGINE Serving on https://172.21.15.100:7150 2024-09-10T18:23:48.531 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: cephadm 2024-09-10T18:23:47.213555+0000 mgr.y (mgr.14118) 6 : cephadm [INF] [10/Sep/2024:18:23:47] ENGINE Bus STARTED 2024-09-10T18:23:48.531 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: cephadm 2024-09-10T18:23:47.217547+0000 mgr.y (mgr.14118) 7 : cephadm [ERR] [10/Sep/2024:18:23:47] ENGINE Error in HTTPServer.serve 2024-09-10T18:23:48.531 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: Traceback (most recent call last): 2024-09-10T18:23:48.531 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: self._connections.run(self.expiration_interval) 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: self._run(expiration_interval) 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: s = self.context.wrap_socket( 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: return self.sslsocket_class._create( 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: self.do_handshake() 2024-09-10T18:23:48.532 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:23:48.533 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: self._sslobj.do_handshake() 2024-09-10T18:23:48.533 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:23:48.533 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:48 smithi100 bash[18597]: cluster 2024-09-10T18:23:47.254693+0000 mon.a (mon.0) 49 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-09-10T18:23:49.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eac6c9640 1 Processor -- start 2024-09-10T18:23:49.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eac6c9640 1 -- start start 2024-09-10T18:23:49.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eac6c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea4105570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:49.138 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eac6c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4100f60 con 0x7f6ea4074c60 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eaa465640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea4105570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eaa465640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea4105570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54140/0 (socket says 172.21.15.100:54140) 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eaa465640 1 -- 172.21.15.100:0/1773191762 learned_addr learned my addr 172.21.15.100:0/1773191762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.111+0000 7f6eaa465640 1 -- 172.21.15.100:0/1773191762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4105ab0 con 0x7f6ea4074c60 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eaa465640 1 --2- 172.21.15.100:0/1773191762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea4105570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009e50 tx=0x7f6e9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2960c4a07a9afb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6ea9463640 1 -- 172.21.15.100:0/1773191762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c031400 con 0x7f6ea4074c60 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6ea9463640 1 -- 172.21.15.100:0/1773191762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f6e9c02d050 con 0x7f6ea4074c60 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6ea9463640 1 -- 172.21.15.100:0/1773191762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c0076e0 con 0x7f6ea4074c60 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 -- 172.21.15.100:0/1773191762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 msgr2=0x7f6ea4105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 --2- 172.21.15.100:0/1773191762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea4105570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009e50 tx=0x7f6e9c009840 comp rx=0 tx=0).stop 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 -- 172.21.15.100:0/1773191762 shutdown_connections 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 --2- 172.21.15.100:0/1773191762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea4105570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:49.139 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 -- 172.21.15.100:0/1773191762 >> 172.21.15.100:0/1773191762 conn(0x7f6ea40fabc0 msgr2=0x7f6ea40fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 -- 172.21.15.100:0/1773191762 shutdown_connections 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 -- 172.21.15.100:0/1773191762 wait complete. 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 Processor -- start 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.115+0000 7f6eac6c9640 1 -- start start 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eac6c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea41114e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eac6c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4111a20 con 0x7f6ea4074c60 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eaa465640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea41114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eaa465640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea41114e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54156/0 (socket says 172.21.15.100:54156) 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eaa465640 1 -- 172.21.15.100:0/2423173385 learned_addr learned my addr 172.21.15.100:0/2423173385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eaa465640 1 -- 172.21.15.100:0/2423173385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c009ae0 con 0x7f6ea4074c60 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eaa465640 1 --2- 172.21.15.100:0/2423173385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea41114e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02abe0 tx=0x7f6e9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6ea2ffd640 1 -- 172.21.15.100:0/2423173385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c0077f0 con 0x7f6ea4074c60 2024-09-10T18:23:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea4111c20 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6ea2ffd640 1 -- 172.21.15.100:0/2423173385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f6e9c007da0 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6ea2ffd640 1 -- 172.21.15.100:0/2423173385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9c045ca0 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea4112090 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6ea2ffd640 1 -- 172.21.15.100:0/2423173385 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f6e9c038040 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.119+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea404b720 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.123+0000 7f6ea2ffd640 1 --2- 172.21.15.100:0/2423173385 >> v2:172.21.15.100:6800/2893280940 conn(0x7f6e90039470 0x7f6e9003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.123+0000 7f6ea2ffd640 1 -- 172.21.15.100:0/2423173385 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f6e9c040030 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.123+0000 7f6ea9c64640 1 --2- 172.21.15.100:0/2423173385 >> v2:172.21.15.100:6800/2893280940 conn(0x7f6e90039470 0x7f6e9003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.123+0000 7f6ea9c64640 1 --2- 172.21.15.100:0/2423173385 >> v2:172.21.15.100:6800/2893280940 conn(0x7f6e90039470 0x7f6e9003b920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c006fd0 tx=0x7f6e8c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.131+0000 7f6ea2ffd640 1 -- 172.21.15.100:0/2423173385 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6e9c03a400 con 0x7f6ea4074c60 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.479+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 --> v2:172.21.15.100:6800/2893280940 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ea4112090 con 0x7f6e90039470 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.495+0000 7f6ea2ffd640 1 -- 172.21.15.100:0/2423173385 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6ea4112090 con 0x7f6e90039470 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 >> v2:172.21.15.100:6800/2893280940 conn(0x7f6e90039470 msgr2=0x7f6e9003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:49.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 --2- 172.21.15.100:0/2423173385 >> v2:172.21.15.100:6800/2893280940 conn(0x7f6e90039470 0x7f6e9003b920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c006fd0 tx=0x7f6e8c006db0 comp rx=0 tx=0).stop 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 msgr2=0x7f6ea41114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 --2- 172.21.15.100:0/2423173385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea41114e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c02abe0 tx=0x7f6e9c009840 comp rx=0 tx=0).stop 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 shutdown_connections 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 --2- 172.21.15.100:0/2423173385 >> v2:172.21.15.100:6800/2893280940 conn(0x7f6e90039470 0x7f6e9003b920 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 --2- 172.21.15.100:0/2423173385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ea4074c60 0x7f6ea41114e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 >> 172.21.15.100:0/2423173385 conn(0x7f6ea40fabc0 msgr2=0x7f6ea40fb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 shutdown_connections 2024-09-10T18:23:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:48.499+0000 7f6eac6c9640 1 -- 172.21.15.100:0/2423173385 wait complete. 2024-09-10T18:23:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:49 smithi100 bash[18597]: audit 2024-09-10T18:23:48.483806+0000 mgr.y (mgr.14118) 8 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:49 smithi100 bash[18597]: audit 2024-09-10T18:23:48.494156+0000 mon.a (mon.0) 50 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:23:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:49 smithi100 bash[18597]: audit 2024-09-10T18:23:48.513559+0000 mon.a (mon.0) 51 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:23:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:50 smithi100 bash[18597]: audit 2024-09-10T18:23:50.685354+0000 mgr.y (mgr.14118) 9 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:51.267 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:23:51.267 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.335+0000 7fd69a0e3640 1 Processor -- start 2024-09-10T18:23:51.267 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.335+0000 7fd69a0e3640 1 -- start start 2024-09-10T18:23:51.267 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.335+0000 7fd69a0e3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694078380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:51.267 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.335+0000 7fd69a0e3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940788c0 con 0x7fd694077f70 2024-09-10T18:23:51.267 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.335+0000 7fd6937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694078380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.335+0000 7fd6937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694078380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34372/0 (socket says 172.21.15.100:34372) 2024-09-10T18:23:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.335+0000 7fd6937fe640 1 -- 172.21.15.100:0/2500111220 learned_addr learned my addr 172.21.15.100:0/2500111220 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd6937fe640 1 -- 172.21.15.100:0/2500111220 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd694078a00 con 0x7fd694077f70 2024-09-10T18:23:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd6937fe640 1 --2- 172.21.15.100:0/2500111220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694078380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009e50 tx=0x7fd68c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe539251480bd712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd6927fc640 1 -- 172.21.15.100:0/2500111220 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c031400 con 0x7fd694077f70 2024-09-10T18:23:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd6927fc640 1 -- 172.21.15.100:0/2500111220 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fd68c02d050 con 0x7fd694077f70 2024-09-10T18:23:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd6927fc640 1 -- 172.21.15.100:0/2500111220 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c0076e0 con 0x7fd694077f70 2024-09-10T18:23:51.269 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd69a0e3640 1 -- 172.21.15.100:0/2500111220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 msgr2=0x7fd694078380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:51.269 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd69a0e3640 1 --2- 172.21.15.100:0/2500111220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694078380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009e50 tx=0x7fd68c009840 comp rx=0 tx=0).stop 2024-09-10T18:23:51.269 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd69a0e3640 1 -- 172.21.15.100:0/2500111220 shutdown_connections 2024-09-10T18:23:51.269 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd69a0e3640 1 --2- 172.21.15.100:0/2500111220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694078380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:51.269 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd69a0e3640 1 -- 172.21.15.100:0/2500111220 >> 172.21.15.100:0/2500111220 conn(0x7fd694075de0 msgr2=0x7fd694076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:51.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd69a0e3640 1 -- 172.21.15.100:0/2500111220 shutdown_connections 2024-09-10T18:23:51.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.339+0000 7fd69a0e3640 1 -- 172.21.15.100:0/2500111220 wait complete. 2024-09-10T18:23:51.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd69a0e3640 1 Processor -- start 2024-09-10T18:23:51.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd69a0e3640 1 -- start start 2024-09-10T18:23:51.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd69a0e3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694115860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:51.270 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd69a0e3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd694115da0 con 0x7fd694077f70 2024-09-10T18:23:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd6937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694115860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd6937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694115860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34384/0 (socket says 172.21.15.100:34384) 2024-09-10T18:23:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd6937fe640 1 -- 172.21.15.100:0/1336336865 learned_addr learned my addr 172.21.15.100:0/1336336865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd6937fe640 1 -- 172.21.15.100:0/1336336865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c009ae0 con 0x7fd694077f70 2024-09-10T18:23:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd6937fe640 1 --2- 172.21.15.100:0/1336336865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694115860 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd68c031660 tx=0x7fd68c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:51.271 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd68bfff640 1 -- 172.21.15.100:0/1336336865 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c0318e0 con 0x7fd694077f70 2024-09-10T18:23:51.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd68bfff640 1 -- 172.21.15.100:0/1336336865 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fd68c03a400 con 0x7fd694077f70 2024-09-10T18:23:51.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd694115fa0 con 0x7fd694077f70 2024-09-10T18:23:51.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd68bfff640 1 -- 172.21.15.100:0/1336336865 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68c0443e0 con 0x7fd694077f70 2024-09-10T18:23:51.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.343+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd694116410 con 0x7fd694077f70 2024-09-10T18:23:51.272 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.347+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69404b720 con 0x7fd694077f70 2024-09-10T18:23:51.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.347+0000 7fd68bfff640 1 -- 172.21.15.100:0/1336336865 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7fd68c038040 con 0x7fd694077f70 2024-09-10T18:23:51.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.347+0000 7fd68bfff640 1 --2- 172.21.15.100:0/1336336865 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd680039470 0x7fd68003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:51.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.347+0000 7fd68bfff640 1 -- 172.21.15.100:0/1336336865 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fd68c040030 con 0x7fd694077f70 2024-09-10T18:23:51.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.347+0000 7fd692ffd640 1 --2- 172.21.15.100:0/1336336865 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd680039470 0x7fd68003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:51.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.347+0000 7fd692ffd640 1 --2- 172.21.15.100:0/1336336865 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd680039470 0x7fd68003b920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd67c006fd0 tx=0x7fd67c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:51.273 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.355+0000 7fd68bfff640 1 -- 172.21.15.100:0/1336336865 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd68c0422f0 con 0x7fd694077f70 2024-09-10T18:23:51.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.679+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 --> v2:172.21.15.100:6800/2893280940 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd694116410 con 0x7fd680039470 2024-09-10T18:23:51.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd68bfff640 1 -- 172.21.15.100:0/1336336865 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd694116410 con 0x7fd680039470 2024-09-10T18:23:51.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd680039470 msgr2=0x7fd68003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:51.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 --2- 172.21.15.100:0/1336336865 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd680039470 0x7fd68003b920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd67c006fd0 tx=0x7fd67c006db0 comp rx=0 tx=0).stop 2024-09-10T18:23:51.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 msgr2=0x7fd694115860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:51.274 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 --2- 172.21.15.100:0/1336336865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694115860 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd68c031660 tx=0x7fd68c009840 comp rx=0 tx=0).stop 2024-09-10T18:23:51.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 shutdown_connections 2024-09-10T18:23:51.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 --2- 172.21.15.100:0/1336336865 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd680039470 0x7fd68003b920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:51.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 --2- 172.21.15.100:0/1336336865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd694077f70 0x7fd694115860 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:51.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 >> 172.21.15.100:0/1336336865 conn(0x7fd694075de0 msgr2=0x7fd694100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:51.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 shutdown_connections 2024-09-10T18:23:51.275 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:50.683+0000 7fd69a0e3640 1 -- 172.21.15.100:0/1336336865 wait complete. 2024-09-10T18:23:51.275 INFO:teuthology.orchestra.run.smithi100.stdout:Generating ssh key... 2024-09-10T18:23:53.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:52 smithi100 bash[18597]: audit 2024-09-10T18:23:52.720155+0000 mgr.y (mgr.14118) 10 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:53.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:52 smithi100 bash[18597]: cephadm 2024-09-10T18:23:52.720556+0000 mgr.y (mgr.14118) 11 : cephadm [INF] Generating ssh key... 2024-09-10T18:23:53.236 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: Generating public/private rsa key pair. 2024-09-10T18:23:53.236 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: Your identification has been saved in /tmp/tmpn7jwn97x/key 2024-09-10T18:23:53.236 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: Your public key has been saved in /tmp/tmpn7jwn97x/key.pub 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: The key fingerprint is: 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: SHA256:iijfb+4hYPZn0XIbsT1Ohqs9s3EyNTREeaxhUHpGkqI ceph-505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: The key's randomart image is: 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: +---[RSA 3072]----+ 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | o==o | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | . .=+ o | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | . .o.=+ | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | E . O.. | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | + o S B | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | o + . = O o | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: |. . + = * o | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | o . +.+o= | 2024-09-10T18:23:53.237 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: | . .== o+ | 2024-09-10T18:23:53.238 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:23:53 smithi100 bash[18863]: +----[SHA256]-----+ 2024-09-10T18:23:53.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3ff01d3640 1 Processor -- start 2024-09-10T18:23:53.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3ff01d3640 1 -- start start 2024-09-10T18:23:53.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3ff01d3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:53.668 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3ff01d3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81031e0 con 0x7f3fe8102890 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3fedf6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3fedf6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34400/0 (socket says 172.21.15.100:34400) 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3fedf6f640 1 -- 172.21.15.100:0/1575913513 learned_addr learned my addr 172.21.15.100:0/1575913513 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.371+0000 7f3fedf6f640 1 -- 172.21.15.100:0/1575913513 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8103320 con 0x7f3fe8102890 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3fedf6f640 1 --2- 172.21.15.100:0/1575913513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8102ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009840 tx=0x7f3fdc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bce12331cea40c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3fecf6d640 1 -- 172.21.15.100:0/1575913513 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc02add0 con 0x7f3fe8102890 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3fecf6d640 1 -- 172.21.15.100:0/1575913513 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f3fdc032890 con 0x7f3fe8102890 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3fecf6d640 1 -- 172.21.15.100:0/1575913513 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc0317a0 con 0x7f3fe8102890 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3ff01d3640 1 -- 172.21.15.100:0/1575913513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 msgr2=0x7f3fe8102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3ff01d3640 1 --2- 172.21.15.100:0/1575913513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8102ca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009840 tx=0x7f3fdc0091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3ff01d3640 1 -- 172.21.15.100:0/1575913513 shutdown_connections 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3ff01d3640 1 --2- 172.21.15.100:0/1575913513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8102ca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:53.669 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3ff01d3640 1 -- 172.21.15.100:0/1575913513 >> 172.21.15.100:0/1575913513 conn(0x7f3fe8075f80 msgr2=0x7f3fe8076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3ff01d3640 1 -- 172.21.15.100:0/1575913513 shutdown_connections 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.375+0000 7f3ff01d3640 1 -- 172.21.15.100:0/1575913513 wait complete. 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3ff01d3640 1 Processor -- start 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3ff01d3640 1 -- start start 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3ff01d3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8111460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3ff01d3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81119a0 con 0x7f3fe8102890 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fedf6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8111460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fedf6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8111460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34402/0 (socket says 172.21.15.100:34402) 2024-09-10T18:23:53.670 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fedf6f640 1 -- 172.21.15.100:0/3071359341 learned_addr learned my addr 172.21.15.100:0/3071359341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fedf6f640 1 -- 172.21.15.100:0/3071359341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc009450 con 0x7f3fe8102890 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fedf6f640 1 --2- 172.21.15.100:0/3071359341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8111460 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0313e0 tx=0x7f3fdc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fe67fc640 1 -- 172.21.15.100:0/3071359341 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc031e70 con 0x7f3fe8102890 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe8111ba0 con 0x7f3fe8102890 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fe67fc640 1 -- 172.21.15.100:0/3071359341 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f3fdc03c850 con 0x7f3fe8102890 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe8112010 con 0x7f3fe8102890 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fe67fc640 1 -- 172.21.15.100:0/3071359341 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc03a510 con 0x7f3fe8102890 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fe67fc640 1 -- 172.21.15.100:0/3071359341 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f3fdc03a670 con 0x7f3fe8102890 2024-09-10T18:23:53.671 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fe67fc640 1 --2- 172.21.15.100:0/3071359341 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3fd0039430 0x7f3fd003b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.379+0000 7f3fe67fc640 1 -- 172.21.15.100:0/3071359341 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f3fdc06d460 con 0x7f3fe8102890 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.383+0000 7f3fed76e640 1 --2- 172.21.15.100:0/3071359341 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3fd0039430 0x7f3fd003b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.383+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe805ed70 con 0x7f3fe8102890 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.383+0000 7f3fed76e640 1 --2- 172.21.15.100:0/3071359341 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3fd0039430 0x7f3fd003b8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006fd0 tx=0x7f3fd8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.387+0000 7f3fe67fc640 1 -- 172.21.15.100:0/3071359341 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3fdc037070 con 0x7f3fe8102890 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:52.715+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 --> v2:172.21.15.100:6800/2893280940 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fe81053a0 con 0x7f3fd0039430 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.035+0000 7f3fe67fc640 1 -- 172.21.15.100:0/3071359341 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3fe81053a0 con 0x7f3fd0039430 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3fd0039430 msgr2=0x7f3fd003b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 --2- 172.21.15.100:0/3071359341 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3fd0039430 0x7f3fd003b8e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006fd0 tx=0x7f3fd8006db0 comp rx=0 tx=0).stop 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 msgr2=0x7f3fe8111460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 --2- 172.21.15.100:0/3071359341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8111460 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0313e0 tx=0x7f3fdc0091d0 comp rx=0 tx=0).stop 2024-09-10T18:23:53.672 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 shutdown_connections 2024-09-10T18:23:53.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 --2- 172.21.15.100:0/3071359341 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3fd0039430 0x7f3fd003b8e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:53.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 --2- 172.21.15.100:0/3071359341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fe8102890 0x7f3fe8111460 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:53.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 >> 172.21.15.100:0/3071359341 conn(0x7f3fe8075f80 msgr2=0x7f3fe8104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:53.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 shutdown_connections 2024-09-10T18:23:53.673 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:53.039+0000 7f3ff01d3640 1 -- 172.21.15.100:0/3071359341 wait complete. 2024-09-10T18:23:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:54 smithi100 bash[18597]: audit 2024-09-10T18:23:53.032485+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:23:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:54 smithi100 bash[18597]: audit 2024-09-10T18:23:53.037785+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:23:55.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDToza/hxKAXIIBofZJbo+7MnC9mvD43Cr+oSLJV/7UaF5vYpyC1mumsSIvTzUZ5aK40GVZP7isbj7LG6MqnGlXsCGD8QqEPkNvVDE6Tx9D8cD5eqhCFLDL6osaxD2sJEo4cWQEYjh6/Pjq9JRUUcpda9ooxmO5I5vDQBKGJqHhMJQJW/IeGA93mcnYaMI45bYG9dW3qcZUI1rWwuFRNmg6XgRCcS3kz3OjtzF3EJU9s5kqObujt/XRrs567Qk581U5g+hq4rUqSEogDyTtEvO5xJPfZOgeuuG9Q9JFdmECCUVgehJjVogJKckC5a/+qZ/wfMtQlOI+ADe5WV1Th1enFGjqSWk32ZijHckw3lKjEo3QSzBPNqOp7/uSmMAtmbReqCygoCaVMob1sO1qnNg6p4ZZfGLrYSXxNFToQs5Oc8glYUohTZwxHmtMCYC7UwVwE24qitVxzWkOGvl++gr2JyxAvBotOh1BL6t2w2BWKlW8DGgSeobMficvWBsAaWs= ceph-505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:23:55.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.759+0000 7f1a401d2640 1 Processor -- start 2024-09-10T18:23:55.717 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.759+0000 7f1a401d2640 1 -- start start 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a401d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a401d2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38103160 con 0x7f1a38102810 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3df6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3df6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34406/0 (socket says 172.21.15.100:34406) 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3df6e640 1 -- 172.21.15.100:0/2986488373 learned_addr learned my addr 172.21.15.100:0/2986488373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3df6e640 1 -- 172.21.15.100:0/2986488373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a381032a0 con 0x7f1a38102810 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3df6e640 1 --2- 172.21.15.100:0/2986488373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38102c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009e50 tx=0x7f1a28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d04985c3d6210 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3cf6c640 1 -- 172.21.15.100:0/2986488373 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a28031400 con 0x7f1a38102810 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3cf6c640 1 -- 172.21.15.100:0/2986488373 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f1a2802d050 con 0x7f1a38102810 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.763+0000 7f1a3cf6c640 1 -- 172.21.15.100:0/2986488373 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a280076e0 con 0x7f1a38102810 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- 172.21.15.100:0/2986488373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 msgr2=0x7f1a38102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 --2- 172.21.15.100:0/2986488373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38102c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a28009e50 tx=0x7f1a28009840 comp rx=0 tx=0).stop 2024-09-10T18:23:55.718 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- 172.21.15.100:0/2986488373 shutdown_connections 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 --2- 172.21.15.100:0/2986488373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38102c20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- 172.21.15.100:0/2986488373 >> 172.21.15.100:0/2986488373 conn(0x7f1a38075de0 msgr2=0x7f1a38076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- 172.21.15.100:0/2986488373 shutdown_connections 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- 172.21.15.100:0/2986488373 wait complete. 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 Processor -- start 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- start start 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38111440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38111980 con 0x7f1a38102810 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a3df6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38111440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a3df6e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38111440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34422/0 (socket says 172.21.15.100:34422) 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a3df6e640 1 -- 172.21.15.100:0/1576062003 learned_addr learned my addr 172.21.15.100:0/1576062003 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a3df6e640 1 -- 172.21.15.100:0/1576062003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28009ae0 con 0x7f1a38102810 2024-09-10T18:23:55.719 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a3df6e640 1 --2- 172.21.15.100:0/1576062003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38111440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a280070b0 tx=0x7f1a28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a367fc640 1 -- 172.21.15.100:0/1576062003 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a28007b60 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a367fc640 1 -- 172.21.15.100:0/1576062003 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7f1a28007cc0 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a367fc640 1 -- 172.21.15.100:0/1576062003 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a28045e30 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a38111b80 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.767+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a38111ff0 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.771+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3804b720 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.771+0000 7f1a367fc640 1 -- 172.21.15.100:0/1576062003 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f1a28038040 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.771+0000 7f1a367fc640 1 --2- 172.21.15.100:0/1576062003 >> v2:172.21.15.100:6800/2893280940 conn(0x7f1a20039470 0x7f1a2003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.771+0000 7f1a367fc640 1 -- 172.21.15.100:0/1576062003 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f1a28040030 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.771+0000 7f1a3d76d640 1 --2- 172.21.15.100:0/1576062003 >> v2:172.21.15.100:6800/2893280940 conn(0x7f1a20039470 0x7f1a2003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.771+0000 7f1a3d76d640 1 --2- 172.21.15.100:0/1576062003 >> v2:172.21.15.100:6800/2893280940 conn(0x7f1a20039470 0x7f1a2003b920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006fd0 tx=0x7f1a2c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:54.779+0000 7f1a367fc640 1 -- 172.21.15.100:0/1576062003 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f1a28038d60 con 0x7f1a38102810 2024-09-10T18:23:55.720 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.107+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 --> v2:172.21.15.100:6800/2893280940 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a38111ff0 con 0x7f1a20039470 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.107+0000 7f1a367fc640 1 -- 172.21.15.100:0/1576062003 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1a38111ff0 con 0x7f1a20039470 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 >> v2:172.21.15.100:6800/2893280940 conn(0x7f1a20039470 msgr2=0x7f1a2003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 --2- 172.21.15.100:0/1576062003 >> v2:172.21.15.100:6800/2893280940 conn(0x7f1a20039470 0x7f1a2003b920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c006fd0 tx=0x7f1a2c006db0 comp rx=0 tx=0).stop 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 msgr2=0x7f1a38111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 --2- 172.21.15.100:0/1576062003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38111440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a280070b0 tx=0x7f1a28009840 comp rx=0 tx=0).stop 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 shutdown_connections 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 --2- 172.21.15.100:0/1576062003 >> v2:172.21.15.100:6800/2893280940 conn(0x7f1a20039470 0x7f1a2003b920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 --2- 172.21.15.100:0/1576062003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a38102810 0x7f1a38111440 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 >> 172.21.15.100:0/1576062003 conn(0x7f1a38075de0 msgr2=0x7f1a38104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 shutdown_connections 2024-09-10T18:23:55.721 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:55.111+0000 7f1a401d2640 1 -- 172.21.15.100:0/1576062003 wait complete. 2024-09-10T18:23:55.722 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:23:55.722 INFO:teuthology.orchestra.run.smithi100.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:23:55.722 INFO:teuthology.orchestra.run.smithi100.stdout:Adding host smithi100... 2024-09-10T18:23:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:55 smithi100 bash[18597]: audit 2024-09-10T18:23:55.111297+0000 mgr.y (mgr.14118) 12 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:58.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:57 smithi100 bash[18597]: audit 2024-09-10T18:23:57.186661+0000 mgr.y (mgr.14118) 13 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:23:59.191 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:23:58 smithi100 bash[18597]: cephadm 2024-09-10T18:23:58.391226+0000 mgr.y (mgr.14118) 14 : cephadm [INF] Deploying cephadm binary to smithi100 2024-09-10T18:24:01.853 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Added host 'smithi100' with addr '172.21.15.100' 2024-09-10T18:24:01.854 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.831+0000 7fa0bb5ce640 1 Processor -- start 2024-09-10T18:24:01.854 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.831+0000 7fa0bb5ce640 1 -- start start 2024-09-10T18:24:01.854 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.831+0000 7fa0bb5ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:01.854 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.831+0000 7fa0bb5ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4103160 con 0x7fa0b4102810 2024-09-10T18:24:01.854 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b936a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b936a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34438/0 (socket says 172.21.15.100:34438) 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b936a640 1 -- 172.21.15.100:0/2274242461 learned_addr learned my addr 172.21.15.100:0/2274242461 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b936a640 1 -- 172.21.15.100:0/2274242461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b41032a0 con 0x7fa0b4102810 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b936a640 1 --2- 172.21.15.100:0/2274242461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4102c20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009e50 tx=0x7fa0ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f128b9fa8cf4bb6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b3fff640 1 -- 172.21.15.100:0/2274242461 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac031400 con 0x7fa0b4102810 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b3fff640 1 -- 172.21.15.100:0/2274242461 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fa0ac02d050 con 0x7fa0b4102810 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0b3fff640 1 -- 172.21.15.100:0/2274242461 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac0076e0 con 0x7fa0b4102810 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/2274242461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 msgr2=0x7fa0b4102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0bb5ce640 1 --2- 172.21.15.100:0/2274242461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4102c20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009e50 tx=0x7fa0ac009840 comp rx=0 tx=0).stop 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/2274242461 shutdown_connections 2024-09-10T18:24:01.857 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0bb5ce640 1 --2- 172.21.15.100:0/2274242461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4102c20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/2274242461 >> 172.21.15.100:0/2274242461 conn(0x7fa0b4075de0 msgr2=0x7fa0b4076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/2274242461 shutdown_connections 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.835+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/2274242461 wait complete. 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0bb5ce640 1 Processor -- start 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0bb5ce640 1 -- start start 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0bb5ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4111440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0bb5ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b4111980 con 0x7fa0b4102810 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b936a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4111440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b936a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4111440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34450/0 (socket says 172.21.15.100:34450) 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b936a640 1 -- 172.21.15.100:0/4256999031 learned_addr learned my addr 172.21.15.100:0/4256999031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b936a640 1 -- 172.21.15.100:0/4256999031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac009ae0 con 0x7fa0b4102810 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b936a640 1 --2- 172.21.15.100:0/4256999031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4111440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac02abe0 tx=0x7fa0ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:01.858 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b1ffb640 1 -- 172.21.15.100:0/4256999031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac0077f0 con 0x7fa0b4102810 2024-09-10T18:24:01.859 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b1ffb640 1 -- 172.21.15.100:0/4256999031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fa0ac007da0 con 0x7fa0b4102810 2024-09-10T18:24:01.859 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b4111b80 con 0x7fa0b4102810 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b1ffb640 1 -- 172.21.15.100:0/4256999031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ac045cd0 con 0x7fa0b4102810 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b4111ff0 con 0x7fa0b4102810 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b1ffb640 1 -- 172.21.15.100:0/4256999031 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7fa0ac038040 con 0x7fa0b4102810 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b1ffb640 1 --2- 172.21.15.100:0/4256999031 >> v2:172.21.15.100:6800/2893280940 conn(0x7fa09c039470 0x7fa09c03b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b1ffb640 1 -- 172.21.15.100:0/4256999031 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fa0ac040030 con 0x7fa0b4102810 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.839+0000 7fa0b8b69640 1 --2- 172.21.15.100:0/4256999031 >> v2:172.21.15.100:6800/2893280940 conn(0x7fa09c039470 0x7fa09c03b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.843+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b405ed70 con 0x7fa0b4102810 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.843+0000 7fa0b8b69640 1 --2- 172.21.15.100:0/4256999031 >> v2:172.21.15.100:6800/2893280940 conn(0x7fa09c039470 0x7fa09c03b920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8006fd0 tx=0x7fa0a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:56.847+0000 7fa0b1ffb640 1 -- 172.21.15.100:0/4256999031 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa0ac036030 con 0x7fa0b4102810 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:23:57.179+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 --> v2:172.21.15.100:6800/2893280940 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0b4105370 con 0x7fa09c039470 2024-09-10T18:24:01.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.963+0000 7fa0b1ffb640 1 -- 172.21.15.100:0/4256999031 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa0b4105370 con 0x7fa09c039470 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 >> v2:172.21.15.100:6800/2893280940 conn(0x7fa09c039470 msgr2=0x7fa09c03b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 --2- 172.21.15.100:0/4256999031 >> v2:172.21.15.100:6800/2893280940 conn(0x7fa09c039470 0x7fa09c03b920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8006fd0 tx=0x7fa0a8006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 msgr2=0x7fa0b4111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 --2- 172.21.15.100:0/4256999031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4111440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac02abe0 tx=0x7fa0ac009840 comp rx=0 tx=0).stop 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 shutdown_connections 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 --2- 172.21.15.100:0/4256999031 >> v2:172.21.15.100:6800/2893280940 conn(0x7fa09c039470 0x7fa09c03b920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 --2- 172.21.15.100:0/4256999031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0b4102810 0x7fa0b4111440 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 >> 172.21.15.100:0/4256999031 conn(0x7fa0b4075de0 msgr2=0x7fa0b4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 shutdown_connections 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:00.967+0000 7fa0bb5ce640 1 -- 172.21.15.100:0/4256999031 wait complete. 2024-09-10T18:24:01.862 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mon service... 2024-09-10T18:24:02.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:01 smithi100 bash[18597]: audit 2024-09-10T18:24:00.966511+0000 mon.a (mon.0) 54 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:24:02.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:01 smithi100 bash[18597]: cephadm 2024-09-10T18:24:00.967161+0000 mgr.y (mgr.14118) 15 : cephadm [INF] Added host smithi100 2024-09-10T18:24:02.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:01 smithi100 bash[18597]: audit 2024-09-10T18:24:00.967625+0000 mon.a (mon.0) 55 : audit [DBG] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:24:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.507+0000 7fd182073640 1 Processor -- start 2024-09-10T18:24:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.507+0000 7fd182073640 1 -- start start 2024-09-10T18:24:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.507+0000 7fd182073640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c103130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.507+0000 7fd182073640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c103670 con 0x7fd17c074ca0 2024-09-10T18:24:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.507+0000 7fd17b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:04.569 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.507+0000 7fd17b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33054/0 (socket says 172.21.15.100:33054) 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.507+0000 7fd17b7fe640 1 -- 172.21.15.100:0/3884825286 learned_addr learned my addr 172.21.15.100:0/3884825286 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd17b7fe640 1 -- 172.21.15.100:0/3884825286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c1037b0 con 0x7fd17c074ca0 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd17b7fe640 1 --2- 172.21.15.100:0/3884825286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c103130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009840 tx=0x7fd16c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7613e1364dffbe9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd17a7fc640 1 -- 172.21.15.100:0/3884825286 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd16c02add0 con 0x7fd17c074ca0 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd17a7fc640 1 -- 172.21.15.100:0/3884825286 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fd16c032890 con 0x7fd17c074ca0 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd17a7fc640 1 -- 172.21.15.100:0/3884825286 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd16c0317a0 con 0x7fd17c074ca0 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 -- 172.21.15.100:0/3884825286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 msgr2=0x7fd17c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 --2- 172.21.15.100:0/3884825286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c103130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009840 tx=0x7fd16c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 -- 172.21.15.100:0/3884825286 shutdown_connections 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 --2- 172.21.15.100:0/3884825286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c103130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 -- 172.21.15.100:0/3884825286 >> 172.21.15.100:0/3884825286 conn(0x7fd17c0fabc0 msgr2=0x7fd17c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 -- 172.21.15.100:0/3884825286 shutdown_connections 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 -- 172.21.15.100:0/3884825286 wait complete. 2024-09-10T18:24:04.570 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 Processor -- start 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.511+0000 7fd182073640 1 -- start start 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd182073640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c10f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd182073640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c10f7f0 con 0x7fd17c074ca0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd17b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c10f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd17b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c10f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33064/0 (socket says 172.21.15.100:33064) 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd17b7fe640 1 -- 172.21.15.100:0/3506037945 learned_addr learned my addr 172.21.15.100:0/3506037945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd17b7fe640 1 -- 172.21.15.100:0/3506037945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c009450 con 0x7fd17c074ca0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd17b7fe640 1 --2- 172.21.15.100:0/3506037945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c10f2b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0313e0 tx=0x7fd16c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd163fff640 1 -- 172.21.15.100:0/3506037945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd16c032cd0 con 0x7fd17c074ca0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c10f9f0 con 0x7fd17c074ca0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd163fff640 1 -- 172.21.15.100:0/3506037945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 988+0+0 (secure 0 0 0) 0x7fd16c03c850 con 0x7fd17c074ca0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd163fff640 1 -- 172.21.15.100:0/3506037945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd16c046480 con 0x7fd17c074ca0 2024-09-10T18:24:04.571 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c10fe60 con 0x7fd17c074ca0 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd163fff640 1 -- 172.21.15.100:0/3506037945 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7fd16c03c400 con 0x7fd17c074ca0 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd163fff640 1 --2- 172.21.15.100:0/3506037945 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd164039430 0x7fd16403b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd163fff640 1 -- 172.21.15.100:0/3506037945 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fd16c06c180 con 0x7fd17c074ca0 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.515+0000 7fd17affd640 1 --2- 172.21.15.100:0/3506037945 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd164039430 0x7fd16403b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.519+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c05ed70 con 0x7fd17c074ca0 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.519+0000 7fd17affd640 1 --2- 172.21.15.100:0/3506037945 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd164039430 0x7fd16403b8e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.527+0000 7fd163fff640 1 -- 172.21.15.100:0/3506037945 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd16c037070 con 0x7fd17c074ca0 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.871+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 --> v2:172.21.15.100:6800/2893280940 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd17c0fbec0 con 0x7fd164039430 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd163fff640 1 -- 172.21.15.100:0/3506037945 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd17c0fbec0 con 0x7fd164039430 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd164039430 msgr2=0x7fd16403b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 --2- 172.21.15.100:0/3506037945 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd164039430 0x7fd16403b8e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd170006fd0 tx=0x7fd170006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 msgr2=0x7fd17c10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:04.572 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 --2- 172.21.15.100:0/3506037945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c10f2b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0313e0 tx=0x7fd16c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 shutdown_connections 2024-09-10T18:24:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 --2- 172.21.15.100:0/3506037945 >> v2:172.21.15.100:6800/2893280940 conn(0x7fd164039430 0x7fd16403b8e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 --2- 172.21.15.100:0/3506037945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd17c074ca0 0x7fd17c10f2b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 >> 172.21.15.100:0/3506037945 conn(0x7fd17c0fabc0 msgr2=0x7fd17c0fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 shutdown_connections 2024-09-10T18:24:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:03.883+0000 7fd182073640 1 -- 172.21.15.100:0/3506037945 wait complete. 2024-09-10T18:24:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mgr service... 2024-09-10T18:24:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:04 smithi100 bash[18597]: audit 2024-09-10T18:24:03.878800+0000 mgr.y (mgr.14118) 16 : audit [DBG] from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:24:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:04 smithi100 bash[18597]: cephadm 2024-09-10T18:24:03.882045+0000 mgr.y (mgr.14118) 17 : cephadm [INF] Saving service mon spec with placement count:5 2024-09-10T18:24:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:04 smithi100 bash[18597]: audit 2024-09-10T18:24:03.885315+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:24:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:04 smithi100 bash[18597]: audit 2024-09-10T18:24:04.178454+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:24:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:04 smithi100 bash[18597]: audit 2024-09-10T18:24:04.492655+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:24:06.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:06 smithi100 bash[18597]: cluster 2024-09-10T18:24:05.189231+0000 mgr.y (mgr.14118) 18 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:06.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:24:06.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f2124434640 1 Processor -- start 2024-09-10T18:24:06.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f2124434640 1 -- start start 2024-09-10T18:24:06.920 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f2124434640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f2124434640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c103210 con 0x7f211c1028c0 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f21221d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f21221d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33074/0 (socket says 172.21.15.100:33074) 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f21221d0640 1 -- 172.21.15.100:0/3662487724 learned_addr learned my addr 172.21.15.100:0/3662487724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.867+0000 7f21221d0640 1 -- 172.21.15.100:0/3662487724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c103350 con 0x7f211c1028c0 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f21221d0640 1 --2- 172.21.15.100:0/3662487724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c102cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f210c009840 tx=0x7f210c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776bfaae467f84af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f21211ce640 1 -- 172.21.15.100:0/3662487724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c02ae60 con 0x7f211c1028c0 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f21211ce640 1 -- 172.21.15.100:0/3662487724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f210c0328b0 con 0x7f211c1028c0 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f21211ce640 1 -- 172.21.15.100:0/3662487724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c031740 con 0x7f211c1028c0 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 -- 172.21.15.100:0/3662487724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 msgr2=0x7f211c102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 --2- 172.21.15.100:0/3662487724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c102cd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f210c009840 tx=0x7f210c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:06.921 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 -- 172.21.15.100:0/3662487724 shutdown_connections 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 --2- 172.21.15.100:0/3662487724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c102cd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 -- 172.21.15.100:0/3662487724 >> 172.21.15.100:0/3662487724 conn(0x7f211c075f20 msgr2=0x7f211c076360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 -- 172.21.15.100:0/3662487724 shutdown_connections 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 -- 172.21.15.100:0/3662487724 wait complete. 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 Processor -- start 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 -- start start 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c111450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f2124434640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c111990 con 0x7f211c1028c0 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f21221d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c111450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f21221d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c111450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33090/0 (socket says 172.21.15.100:33090) 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.871+0000 7f21221d0640 1 -- 172.21.15.100:0/2731078555 learned_addr learned my addr 172.21.15.100:0/2731078555 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f21221d0640 1 -- 172.21.15.100:0/2731078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c009450 con 0x7f211c1028c0 2024-09-10T18:24:06.922 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f21221d0640 1 --2- 172.21.15.100:0/2731078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c111450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f210c032670 tx=0x7f210c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2112ffd640 1 -- 172.21.15.100:0/2731078555 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c031c40 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2112ffd640 1 -- 172.21.15.100:0/2731078555 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f210c031da0 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2112ffd640 1 -- 172.21.15.100:0/2731078555 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c03bea0 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c111b90 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c112000 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2112ffd640 1 -- 172.21.15.100:0/2731078555 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f210c039040 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2112ffd640 1 --2- 172.21.15.100:0/2731078555 >> v2:172.21.15.100:6800/2893280940 conn(0x7f21080393e0 0x7f210803b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c04b720 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f2112ffd640 1 -- 172.21.15.100:0/2731078555 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f210c06c5c0 con 0x7f211c1028c0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.875+0000 7f21219cf640 1 --2- 172.21.15.100:0/2731078555 >> v2:172.21.15.100:6800/2893280940 conn(0x7f21080393e0 0x7f210803b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.879+0000 7f21219cf640 1 --2- 172.21.15.100:0/2731078555 >> v2:172.21.15.100:6800/2893280940 conn(0x7f21080393e0 0x7f210803b890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2118006fd0 tx=0x7f2118006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:06.923 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:05.883+0000 7f2112ffd640 1 -- 172.21.15.100:0/2731078555 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f210c037070 con 0x7f211c1028c0 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.231+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 --> v2:172.21.15.100:6800/2893280940 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f211c105330 con 0x7f21080393e0 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.239+0000 7f2112ffd640 1 -- 172.21.15.100:0/2731078555 <== mgr.14118 v2:172.21.15.100:6800/2893280940 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f211c105330 con 0x7f21080393e0 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 >> v2:172.21.15.100:6800/2893280940 conn(0x7f21080393e0 msgr2=0x7f210803b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 --2- 172.21.15.100:0/2731078555 >> v2:172.21.15.100:6800/2893280940 conn(0x7f21080393e0 0x7f210803b890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2118006fd0 tx=0x7f2118006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 msgr2=0x7f211c111450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 --2- 172.21.15.100:0/2731078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c111450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f210c032670 tx=0x7f210c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 shutdown_connections 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 --2- 172.21.15.100:0/2731078555 >> v2:172.21.15.100:6800/2893280940 conn(0x7f21080393e0 0x7f210803b890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 --2- 172.21.15.100:0/2731078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f211c1028c0 0x7f211c111450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 >> 172.21.15.100:0/2731078555 conn(0x7f211c075f20 msgr2=0x7f211c104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 shutdown_connections 2024-09-10T18:24:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:06.243+0000 7f2124434640 1 -- 172.21.15.100:0/2731078555 wait complete. 2024-09-10T18:24:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:07 smithi100 bash[18597]: audit 2024-09-10T18:24:06.235565+0000 mgr.y (mgr.14118) 19 : audit [DBG] from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:24:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:07 smithi100 bash[18597]: cephadm 2024-09-10T18:24:06.236626+0000 mgr.y (mgr.14118) 20 : cephadm [INF] Saving service mgr spec with placement count:2 2024-09-10T18:24:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:07 smithi100 bash[18597]: audit 2024-09-10T18:24:06.243611+0000 mon.a (mon.0) 59 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:24:08.539 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:08 smithi100 bash[18597]: cluster 2024-09-10T18:24:07.189730+0000 mgr.y (mgr.14118) 21 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f3123c21640 1 Processor -- start 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f3123c21640 1 -- start start 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f3123c21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c105640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f3123c21640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c075820 con 0x7f311c075410 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31219bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c105640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31219bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c105640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33096/0 (socket says 172.21.15.100:33096) 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31219bd640 1 -- 172.21.15.100:0/782435997 learned_addr learned my addr 172.21.15.100:0/782435997 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:09.170 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31219bd640 1 -- 172.21.15.100:0/782435997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c101030 con 0x7f311c075410 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31219bd640 1 --2- 172.21.15.100:0/782435997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c105640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3118009840 tx=0x7f31180091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821c85fab4d03aea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31209bb640 1 -- 172.21.15.100:0/782435997 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f311802add0 con 0x7f311c075410 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31209bb640 1 -- 172.21.15.100:0/782435997 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f31180328b0 con 0x7f311c075410 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.151+0000 7f31209bb640 1 -- 172.21.15.100:0/782435997 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3118031690 con 0x7f311c075410 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 -- 172.21.15.100:0/782435997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 msgr2=0x7f311c105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 --2- 172.21.15.100:0/782435997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c105640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3118009840 tx=0x7f31180091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 -- 172.21.15.100:0/782435997 shutdown_connections 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 --2- 172.21.15.100:0/782435997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c105640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 -- 172.21.15.100:0/782435997 >> 172.21.15.100:0/782435997 conn(0x7f311c0fac90 msgr2=0x7f311c0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 -- 172.21.15.100:0/782435997 shutdown_connections 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 -- 172.21.15.100:0/782435997 wait complete. 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 Processor -- start 2024-09-10T18:24:09.171 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 -- start start 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c111470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f3123c21640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1119b0 con 0x7f311c075410 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f31219bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c111470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f31219bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c111470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33104/0 (socket says 172.21.15.100:33104) 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f31219bd640 1 -- 172.21.15.100:0/2457192031 learned_addr learned my addr 172.21.15.100:0/2457192031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f31219bd640 1 -- 172.21.15.100:0/2457192031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118009450 con 0x7f311c075410 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.155+0000 7f31219bd640 1 --2- 172.21.15.100:0/2457192031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c111470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31127fc640 1 -- 172.21.15.100:0/2457192031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3118031c50 con 0x7f311c075410 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c111bb0 con 0x7f311c075410 2024-09-10T18:24:09.172 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31127fc640 1 -- 172.21.15.100:0/2457192031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f3118031db0 con 0x7f311c075410 2024-09-10T18:24:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31127fc640 1 -- 172.21.15.100:0/2457192031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f311803a3a0 con 0x7f311c075410 2024-09-10T18:24:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c112740 con 0x7f311c075410 2024-09-10T18:24:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31127fc640 1 -- 172.21.15.100:0/2457192031 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f3118039070 con 0x7f311c075410 2024-09-10T18:24:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31127fc640 1 --2- 172.21.15.100:0/2457192031 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3108039430 0x7f310803b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31127fc640 1 -- 172.21.15.100:0/2457192031 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f311806d750 con 0x7f311c075410 2024-09-10T18:24:09.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c112080 con 0x7f311c075410 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31211bc640 1 --2- 172.21.15.100:0/2457192031 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3108039430 0x7f310803b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.159+0000 7f31211bc640 1 --2- 172.21.15.100:0/2457192031 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3108039430 0x7f310803b8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f310c006fd0 tx=0x7f310c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.167+0000 7f31127fc640 1 -- 172.21.15.100:0/2457192031 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f311c112080 con 0x7f311c075410 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.495+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f311c112080 con 0x7f311c075410 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.503+0000 7f31127fc640 1 -- 172.21.15.100:0/2457192031 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f311c112080 con 0x7f311c075410 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3108039430 msgr2=0x7f310803b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 --2- 172.21.15.100:0/2457192031 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3108039430 0x7f310803b8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f310c006fd0 tx=0x7f310c006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 msgr2=0x7f311c111470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:09.174 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 --2- 172.21.15.100:0/2457192031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c111470 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118008040 comp rx=0 tx=0).stop 2024-09-10T18:24:09.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 shutdown_connections 2024-09-10T18:24:09.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 --2- 172.21.15.100:0/2457192031 >> v2:172.21.15.100:6800/2893280940 conn(0x7f3108039430 0x7f310803b8e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:09.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 --2- 172.21.15.100:0/2457192031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f311c075410 0x7f311c111470 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:09.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 >> 172.21.15.100:0/2457192031 conn(0x7f311c0fac90 msgr2=0x7f311c0fb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:09.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.507+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 shutdown_connections 2024-09-10T18:24:09.175 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:08.511+0000 7f3123c21640 1 -- 172.21.15.100:0/2457192031 wait complete. 2024-09-10T18:24:09.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:09 smithi100 bash[18597]: audit 2024-09-10T18:24:08.508667+0000 mon.a (mon.0) 60 : audit [INF] from='client.? 172.21.15.100:0/2457192031' entity='client.admin' 2024-09-10T18:24:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:10 smithi100 bash[18597]: cluster 2024-09-10T18:24:09.190031+0000 mgr.y (mgr.14118) 22 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:10 smithi100 bash[18597]: audit 2024-09-10T18:24:09.986990+0000 mon.a (mon.0) 61 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:24:11.393 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f55233ac640 1 Processor -- start 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f55233ac640 1 -- start start 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f55233ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c0ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f55233ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c1003c0 con 0x7f551c0ffa70 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5521148640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c0ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5521148640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c0ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50286/0 (socket says 172.21.15.100:50286) 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5521148640 1 -- 172.21.15.100:0/2329722489 learned_addr learned my addr 172.21.15.100:0/2329722489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5521148640 1 -- 172.21.15.100:0/2329722489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c100500 con 0x7f551c0ffa70 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5521148640 1 --2- 172.21.15.100:0/2329722489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c0ffe80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f550c00baf0 tx=0x7f550c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494d77d154b5e733 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5513fff640 1 -- 172.21.15.100:0/2329722489 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f550c032640 con 0x7f551c0ffa70 2024-09-10T18:24:11.394 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5513fff640 1 -- 172.21.15.100:0/2329722489 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f550c032c10 con 0x7f551c0ffa70 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.423+0000 7f5513fff640 1 -- 172.21.15.100:0/2329722489 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f550c031ae0 con 0x7f551c0ffa70 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- 172.21.15.100:0/2329722489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 msgr2=0x7f551c0ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 --2- 172.21.15.100:0/2329722489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c0ffe80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f550c00baf0 tx=0x7f550c00b460 comp rx=0 tx=0).stop 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- 172.21.15.100:0/2329722489 shutdown_connections 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 --2- 172.21.15.100:0/2329722489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c0ffe80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- 172.21.15.100:0/2329722489 >> 172.21.15.100:0/2329722489 conn(0x7f551c0fa380 msgr2=0x7f551c0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- 172.21.15.100:0/2329722489 shutdown_connections 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- 172.21.15.100:0/2329722489 wait complete. 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 Processor -- start 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- start start 2024-09-10T18:24:11.395 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c10cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c10d4a0 con 0x7f551c0ffa70 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5521148640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c10cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5521148640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c10cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50300/0 (socket says 172.21.15.100:50300) 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5521148640 1 -- 172.21.15.100:0/3037935830 learned_addr learned my addr 172.21.15.100:0/3037935830 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5521148640 1 -- 172.21.15.100:0/3037935830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c00b700 con 0x7f551c0ffa70 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5521148640 1 --2- 172.21.15.100:0/3037935830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c10cf60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b2c0 tx=0x7f550c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5511ffb640 1 -- 172.21.15.100:0/3037935830 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f550c032920 con 0x7f551c0ffa70 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5511ffb640 1 -- 172.21.15.100:0/3037935830 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f550c032a80 con 0x7f551c0ffa70 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c10d6a0 con 0x7f551c0ffa70 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f5511ffb640 1 -- 172.21.15.100:0/3037935830 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f550c045590 con 0x7f551c0ffa70 2024-09-10T18:24:11.396 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.427+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c10db10 con 0x7f551c0ffa70 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.431+0000 7f5511ffb640 1 -- 172.21.15.100:0/3037935830 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f550c03b420 con 0x7f551c0ffa70 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.431+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c04b720 con 0x7f551c0ffa70 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.431+0000 7f5511ffb640 1 --2- 172.21.15.100:0/3037935830 >> v2:172.21.15.100:6800/2893280940 conn(0x7f5508039530 0x7f550803b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.431+0000 7f5511ffb640 1 -- 172.21.15.100:0/3037935830 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f550c06fab0 con 0x7f551c0ffa70 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.431+0000 7f5520947640 1 --2- 172.21.15.100:0/3037935830 >> v2:172.21.15.100:6800/2893280940 conn(0x7f5508039530 0x7f550803b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.431+0000 7f5520947640 1 --2- 172.21.15.100:0/3037935830 >> v2:172.21.15.100:6800/2893280940 conn(0x7f5508039530 0x7f550803b9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5518006fd0 tx=0x7f5518006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.439+0000 7f5511ffb640 1 -- 172.21.15.100:0/3037935830 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f550c04caa0 con 0x7f551c0ffa70 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.767+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f551c10db10 con 0x7f551c0ffa70 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.775+0000 7f5511ffb640 1 -- 172.21.15.100:0/3037935830 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f550c04c300 con 0x7f551c0ffa70 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 >> v2:172.21.15.100:6800/2893280940 conn(0x7f5508039530 msgr2=0x7f550803b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:11.397 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 --2- 172.21.15.100:0/3037935830 >> v2:172.21.15.100:6800/2893280940 conn(0x7f5508039530 0x7f550803b9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5518006fd0 tx=0x7f5518006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 msgr2=0x7f551c10cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 --2- 172.21.15.100:0/3037935830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c10cf60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b2c0 tx=0x7f550c03d040 comp rx=0 tx=0).stop 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 shutdown_connections 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 --2- 172.21.15.100:0/3037935830 >> v2:172.21.15.100:6800/2893280940 conn(0x7f5508039530 0x7f550803b9e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 --2- 172.21.15.100:0/3037935830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0ffa70 0x7f551c10cf60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 >> 172.21.15.100:0/3037935830 conn(0x7f551c0fa380 msgr2=0x7f551c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 shutdown_connections 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:10.779+0000 7f55233ac640 1 -- 172.21.15.100:0/3037935830 wait complete. 2024-09-10T18:24:11.398 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling the dashboard module... 2024-09-10T18:24:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:11 smithi100 bash[18597]: audit 2024-09-10T18:24:10.635425+0000 mon.a (mon.0) 62 : audit [INF] from='mgr.14118 172.21.15.100:0/1018713713' entity='mgr.y' 2024-09-10T18:24:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:11 smithi100 bash[18597]: audit 2024-09-10T18:24:10.780451+0000 mon.a (mon.0) 63 : audit [INF] from='client.? 172.21.15.100:0/3037935830' entity='client.admin' 2024-09-10T18:24:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:12 smithi100 bash[18597]: cluster 2024-09-10T18:24:11.190448+0000 mgr.y (mgr.14118) 23 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:14.063 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:13 smithi100 bash[18597]: audit 2024-09-10T18:24:13.036959+0000 mon.a (mon.0) 64 : audit [INF] from='client.? 172.21.15.100:0/889724274' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:24:14.063 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:13 smithi100 bash[18863]: ignoring --setuser ceph since I am not root 2024-09-10T18:24:14.063 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:13 smithi100 bash[18863]: ignoring --setgroup ceph since I am not root 2024-09-10T18:24:14.063 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:13 smithi100 bash[18863]: debug 2024-09-10T18:24:13.963+0000 7fc041f5fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f44854bd640 1 Processor -- start 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f44854bd640 1 -- start start 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f44854bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f4480100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f44854bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480101010 con 0x7f44801006c0 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f4480100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f4480100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50306/0 (socket says 172.21.15.100:50306) 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447effd640 1 -- 172.21.15.100:0/2664159089 learned_addr learned my addr 172.21.15.100:0/2664159089 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447effd640 1 -- 172.21.15.100:0/2664159089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480101150 con 0x7f44801006c0 2024-09-10T18:24:14.465 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447effd640 1 --2- 172.21.15.100:0/2664159089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f4480100ad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4468009840 tx=0x7f44680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4873b2962601cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447dffb640 1 -- 172.21.15.100:0/2664159089 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f446802ae60 con 0x7f44801006c0 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447dffb640 1 -- 172.21.15.100:0/2664159089 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f44680328b0 con 0x7f44801006c0 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.599+0000 7f447dffb640 1 -- 172.21.15.100:0/2664159089 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4468031740 con 0x7f44801006c0 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- 172.21.15.100:0/2664159089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 msgr2=0x7f4480100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 --2- 172.21.15.100:0/2664159089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f4480100ad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4468009840 tx=0x7f44680091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- 172.21.15.100:0/2664159089 shutdown_connections 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 --2- 172.21.15.100:0/2664159089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f4480100ad0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- 172.21.15.100:0/2664159089 >> 172.21.15.100:0/2664159089 conn(0x7f44800fbea0 msgr2=0x7f44800fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- 172.21.15.100:0/2664159089 shutdown_connections 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- 172.21.15.100:0/2664159089 wait complete. 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 Processor -- start 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- start start 2024-09-10T18:24:14.466 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f448010f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448010f770 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f447effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f448010f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f447effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f448010f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50308/0 (socket says 172.21.15.100:50308) 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f447effd640 1 -- 172.21.15.100:0/889724274 learned_addr learned my addr 172.21.15.100:0/889724274 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f447effd640 1 -- 172.21.15.100:0/889724274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4468009450 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f447effd640 1 --2- 172.21.15.100:0/889724274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f448010f230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4468000c00 tx=0x7f44680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4468031b80 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f4468031ce0 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448010f970 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f446803a390 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.603+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448010fde0 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.607+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45457+0+0 (secure 0 0 0) 0x7f4468039040 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.607+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448004b720 con 0x7f44801006c0 2024-09-10T18:24:14.467 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.607+0000 7f44777fe640 1 --2- 172.21.15.100:0/889724274 >> v2:172.21.15.100:6800/2893280940 conn(0x7f446c039470 0x7f446c03b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.607+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f446806c400 con 0x7f44801006c0 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.607+0000 7f447e7fc640 1 --2- 172.21.15.100:0/889724274 >> v2:172.21.15.100:6800/2893280940 conn(0x7f446c039470 0x7f446c03b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.607+0000 7f447e7fc640 1 --2- 172.21.15.100:0/889724274 >> v2:172.21.15.100:6800/2893280940 conn(0x7f446c039470 0x7f446c03b920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4470006fd0 tx=0x7f4470006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:12.611+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4468030d90 con 0x7f44801006c0 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.031+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f448010fde0 con 0x7f44801006c0 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.787+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 9) v1 ==== 45470+0+0 (secure 0 0 0) 0x7f446804ace0 con 0x7f44801006c0 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.787+0000 7f44777fe640 1 -- 172.21.15.100:0/889724274 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f446806bd70 con 0x7f44801006c0 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.791+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 >> v2:172.21.15.100:6800/2893280940 conn(0x7f446c039470 msgr2=0x7f446c03b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 --2- 172.21.15.100:0/889724274 >> v2:172.21.15.100:6800/2893280940 conn(0x7f446c039470 0x7f446c03b920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4470006fd0 tx=0x7f4470006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 msgr2=0x7f448010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 --2- 172.21.15.100:0/889724274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f448010f230 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4468000c00 tx=0x7f44680091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 shutdown_connections 2024-09-10T18:24:14.468 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 --2- 172.21.15.100:0/889724274 >> v2:172.21.15.100:6800/2893280940 conn(0x7f446c039470 0x7f446c03b920 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:14.469 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 --2- 172.21.15.100:0/889724274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44801006c0 0x7f448010f230 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:14.469 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 >> 172.21.15.100:0/889724274 conn(0x7f44800fbea0 msgr2=0x7f44800fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:14.470 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 shutdown_connections 2024-09-10T18:24:14.470 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:13.795+0000 7f44854bd640 1 -- 172.21.15.100:0/889724274 wait complete. 2024-09-10T18:24:14.834 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:14 smithi100 bash[18597]: audit 2024-09-10T18:24:13.792796+0000 mon.a (mon.0) 65 : audit [INF] from='client.? 172.21.15.100:0/889724274' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:24:14.834 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:14 smithi100 bash[18597]: cluster 2024-09-10T18:24:13.792961+0000 mon.a (mon.0) 66 : cluster [DBG] mgrmap e9: y(active, since 28s) 2024-09-10T18:24:14.834 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:14 smithi100 bash[18863]: debug 2024-09-10T18:24:14.699+0000 7fc041f5fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:24:14.835 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:14 smithi100 bash[18863]: debug 2024-09-10T18:24:14.763+0000 7fc041f5fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:24:15.236 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:14 smithi100 bash[18863]: debug 2024-09-10T18:24:14.831+0000 7fc041f5fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:24:15.274 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:14 smithi100 bash[18863]: debug 2024-09-10T18:24:14.983+0000 7fc041f5fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:24:15.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:15 smithi100 bash[18863]: debug 2024-09-10T18:24:15.231+0000 7fc041f5fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:24:15.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:15 smithi100 bash[18863]: debug 2024-09-10T18:24:15.291+0000 7fc041f5fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:24:15.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:15 smithi100 bash[18863]: debug 2024-09-10T18:24:15.431+0000 7fc041f5fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:24:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:16 smithi100 bash[18597]: audit 2024-09-10T18:24:16.012006+0000 mon.a (mon.0) 67 : audit [DBG] from='client.? 172.21.15.100:0/1654810270' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:24:16.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:16 smithi100 bash[18863]: debug 2024-09-10T18:24:16.087+0000 7fc041f5fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:24:16.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:16 smithi100 bash[18863]: debug 2024-09-10T18:24:16.163+0000 7fc041f5fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:24:16.294 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:16 smithi100 bash[18863]: debug 2024-09-10T18:24:16.239+0000 7fc041f5fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:24:16.690 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:24:16.690 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 Processor -- start 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 -- start start 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c100f90 con 0x7f9c6c1005b0 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c6a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c6a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c1009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50344/0 (socket says 172.21.15.100:50344) 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c6a59c640 1 -- 172.21.15.100:0/572965422 learned_addr learned my addr 172.21.15.100:0/572965422 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c6a59c640 1 -- 172.21.15.100:0/572965422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c1017e0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.691 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c6a59c640 1 --2- 172.21.15.100:0/572965422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c1009c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009840 tx=0x7f9c5c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfd68b88dda63a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c69d9b640 1 -- 172.21.15.100:0/572965422 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c02add0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c69d9b640 1 -- 172.21.15.100:0/572965422 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f9c5c0328b0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c69d9b640 1 -- 172.21.15.100:0/572965422 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c031690 con 0x7f9c6c1005b0 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 -- 172.21.15.100:0/572965422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 msgr2=0x7f9c6c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 --2- 172.21.15.100:0/572965422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c1009c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009840 tx=0x7f9c5c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 -- 172.21.15.100:0/572965422 shutdown_connections 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 --2- 172.21.15.100:0/572965422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c1009c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 -- 172.21.15.100:0/572965422 >> 172.21.15.100:0/572965422 conn(0x7f9c6c0fbcb0 msgr2=0x7f9c6c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 -- 172.21.15.100:0/572965422 shutdown_connections 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.583+0000 7f9c70c1e640 1 -- 172.21.15.100:0/572965422 wait complete. 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c70c1e640 1 Processor -- start 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c70c1e640 1 -- start start 2024-09-10T18:24:16.692 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c70c1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c70c1e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c077160 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c6a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c6a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c076c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50346/0 (socket says 172.21.15.100:50346) 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c6a59c640 1 -- 172.21.15.100:0/1654810270 learned_addr learned my addr 172.21.15.100:0/1654810270 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c6a59c640 1 -- 172.21.15.100:0/1654810270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c009450 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c6a59c640 1 --2- 172.21.15.100:0/1654810270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c076c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c000c00 tx=0x7f9c5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c637fe640 1 -- 172.21.15.100:0/1654810270 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c031c50 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c637fe640 1 -- 172.21.15.100:0/1654810270 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f9c5c031db0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c637fe640 1 -- 172.21.15.100:0/1654810270 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c5c03a3a0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c0751f0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c075660 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c637fe640 1 -- 172.21.15.100:0/1654810270 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) v1 ==== 45470+0+0 (secure 0 0 0) 0x7f9c5c039070 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c637fe640 1 --2- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 0x7f9c5803b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c637fe640 1 -- 172.21.15.100:0/1654810270 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f9c5c06d7d0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.693 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c61d9b640 1 -- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 msgr2=0x7f9c5803b970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2893280940 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c05ed70 con 0x7f9c6c1005b0 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.587+0000 7f9c61d9b640 1 --2- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 0x7f9c5803b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.595+0000 7f9c637fe640 1 -- 172.21.15.100:0/1654810270 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9c5c03f080 con 0x7f9c6c1005b0 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.791+0000 7f9c61d9b640 1 -- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 msgr2=0x7f9c5803b970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2893280940 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:15.791+0000 7f9c61d9b640 1 --2- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 0x7f9c5803b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.007+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9c6c1025d0 con 0x7f9c6c1005b0 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.007+0000 7f9c637fe640 1 -- 172.21.15.100:0/1654810270 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f9c5c037070 con 0x7f9c6c1005b0 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 msgr2=0x7f9c5803b970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 --2- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 0x7f9c5803b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 msgr2=0x7f9c6c076c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 --2- 172.21.15.100:0/1654810270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c076c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c000c00 tx=0x7f9c5c008040 comp rx=0 tx=0).stop 2024-09-10T18:24:16.694 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 shutdown_connections 2024-09-10T18:24:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 --2- 172.21.15.100:0/1654810270 >> v2:172.21.15.100:6800/2893280940 conn(0x7f9c580394c0 0x7f9c5803b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 --2- 172.21.15.100:0/1654810270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c6c1005b0 0x7f9c6c076c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 >> 172.21.15.100:0/1654810270 conn(0x7f9c6c0fbcb0 msgr2=0x7f9c6c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 shutdown_connections 2024-09-10T18:24:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:16.011+0000 7f9c70c1e640 1 -- 172.21.15.100:0/1654810270 wait complete. 2024-09-10T18:24:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-09-10T18:24:16.695 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 9... 2024-09-10T18:24:16.782 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:16 smithi100 bash[18863]: debug 2024-09-10T18:24:16.519+0000 7fc041f5fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:24:16.782 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:16 smithi100 bash[18863]: debug 2024-09-10T18:24:16.655+0000 7fc041f5fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:24:17.044 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:16 smithi100 bash[18863]: debug 2024-09-10T18:24:16.967+0000 7fc041f5fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:24:17.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:17 smithi100 bash[18863]: debug 2024-09-10T18:24:17.179+0000 7fc041f5fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:24:17.662 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:17 smithi100 bash[18863]: debug 2024-09-10T18:24:17.595+0000 7fc041f5fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:24:17.884 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:17 smithi100 bash[18863]: debug 2024-09-10T18:24:17.659+0000 7fc041f5fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:24:17.884 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:17 smithi100 bash[18863]: debug 2024-09-10T18:24:17.723+0000 7fc041f5fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:24:17.885 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:17 smithi100 bash[18863]: debug 2024-09-10T18:24:17.815+0000 7fc041f5fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:24:18.178 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:17 smithi100 bash[18863]: debug 2024-09-10T18:24:17.879+0000 7fc041f5fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:24:18.178 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:17 smithi100 bash[18863]: debug 2024-09-10T18:24:17.955+0000 7fc041f5fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:24:18.178 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:18 smithi100 bash[18863]: debug 2024-09-10T18:24:18.099+0000 7fc041f5fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:24:18.433 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:18 smithi100 bash[18863]: debug 2024-09-10T18:24:18.175+0000 7fc041f5fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:24:19.179 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18863]: debug 2024-09-10T18:24:19.119+0000 7fc041f5fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:24:19.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: cluster 2024-09-10T18:24:19.124767+0000 mon.a (mon.0) 68 : cluster [INF] Active manager daemon y restarted 2024-09-10T18:24:19.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: cluster 2024-09-10T18:24:19.125544+0000 mon.a (mon.0) 69 : cluster [INF] Activating manager daemon y 2024-09-10T18:24:19.376 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: cluster 2024-09-10T18:24:19.129907+0000 mon.a (mon.0) 70 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: cluster 2024-09-10T18:24:19.130028+0000 mon.a (mon.0) 71 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00486309s) 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: audit 2024-09-10T18:24:19.133392+0000 mon.a (mon.0) 72 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: audit 2024-09-10T18:24:19.135560+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: audit 2024-09-10T18:24:19.136470+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: audit 2024-09-10T18:24:19.136673+0000 mon.a (mon.0) 75 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: audit 2024-09-10T18:24:19.136816+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: cluster 2024-09-10T18:24:19.146784+0000 mon.a (mon.0) 77 : cluster [INF] Manager daemon y is now available 2024-09-10T18:24:19.377 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:19 smithi100 bash[18597]: audit 2024-09-10T18:24:19.170408+0000 mon.a (mon.0) 78 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18597]: audit 2024-09-10T18:24:19.197164+0000 mon.a (mon.0) 79 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:24:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18597]: audit 2024-09-10T18:24:19.203090+0000 mon.a (mon.0) 80 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:24:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18597]: cluster 2024-09-10T18:24:20.135954+0000 mon.a (mon.0) 81 : cluster [DBG] mgrmap e11: y(active, since 1.01078s) 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: debug 2024-09-10T18:24:20.235+0000 7fbff2ab0640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:24:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: Traceback (most recent call last): 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: self._connections.run(self.expiration_interval) 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: self._run(expiration_interval) 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:24:20.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: s = self.context.wrap_socket( 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: return self.sslsocket_class._create( 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: self.do_handshake() 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: self._sslobj.do_handshake() 2024-09-10T18:24:20.546 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:24:20 smithi100 bash[18863]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:24:20.823 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:24:20.823 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:24:20.823 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 Processor -- start 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 -- start start 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf8100a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8100f60 con 0x7fcdf8100610 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf8100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf8100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50350/0 (socket says 172.21.15.100:50350) 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf7fff640 1 -- 172.21.15.100:0/3155023309 learned_addr learned my addr 172.21.15.100:0/3155023309 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf7fff640 1 -- 172.21.15.100:0/3155023309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf81010a0 con 0x7fcdf8100610 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf7fff640 1 --2- 172.21.15.100:0/3155023309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf8100a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009840 tx=0x7fcdec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae007e910ceb8896 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:20.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf6ffd640 1 -- 172.21.15.100:0/3155023309 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdec02add0 con 0x7fcdf8100610 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf6ffd640 1 -- 172.21.15.100:0/3155023309 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fcdec0328b0 con 0x7fcdf8100610 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdf6ffd640 1 -- 172.21.15.100:0/3155023309 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdec031690 con 0x7fcdf8100610 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 -- 172.21.15.100:0/3155023309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 msgr2=0x7fcdf8100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 --2- 172.21.15.100:0/3155023309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf8100a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009840 tx=0x7fcdec0091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 -- 172.21.15.100:0/3155023309 shutdown_connections 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 --2- 172.21.15.100:0/3155023309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf8100a20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.647+0000 7fcdfe62a640 1 -- 172.21.15.100:0/3155023309 >> 172.21.15.100:0/3155023309 conn(0x7fcdf80fbdd0 msgr2=0x7fcdf80fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 -- 172.21.15.100:0/3155023309 shutdown_connections 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 -- 172.21.15.100:0/3155023309 wait complete. 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 Processor -- start 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 -- start start 2024-09-10T18:24:20.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf810f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf810f800 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf810f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf810f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50352/0 (socket says 172.21.15.100:50352) 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf7fff640 1 -- 172.21.15.100:0/4250424966 learned_addr learned my addr 172.21.15.100:0/4250424966 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf7fff640 1 -- 172.21.15.100:0/4250424966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec009450 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf7fff640 1 --2- 172.21.15.100:0/4250424966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf810f2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcdec035040 tx=0x7fcdec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdec040070 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf810fa00 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf810fe70 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fcdec031ca0 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcdec03a3b0 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.651+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) v1 ==== 45470+0+0 (secure 0 0 0) 0x7fcdec03a510 con 0x7fcdf8100610 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.655+0000 7fcdf4ff9640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 0x7fcde003b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:20.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.655+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fcdec06d020 con 0x7fcdf8100610 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.655+0000 7fcdf77fe640 1 -- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 msgr2=0x7fcde003b970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2893280940 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.655+0000 7fcdf77fe640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 0x7fcde003b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.655+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 --> v2:172.21.15.100:6800/2893280940 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcde4000d40 con 0x7fcde00394c0 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.859+0000 7fcdf77fe640 1 -- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 msgr2=0x7fcde003b970 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2893280940 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:18.859+0000 7fcdf77fe640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 0x7fcde003b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:19.127+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 10) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fcdec043d20 con 0x7fcdf8100610 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:19.127+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 msgr2=0x7fcde003b970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:19.127+0000 7fcdf4ff9640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 0x7fcde003b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.131+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 11) v1 ==== 45364+0+0 (secure 0 0 0) 0x7fcdec03ad30 con 0x7fcdf8100610 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.131+0000 7fcdf4ff9640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcde003cd20 0x7fcde003f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.131+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 --> v2:172.21.15.100:6800/3986579393 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcde4000d40 con 0x7fcde003cd20 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.131+0000 7fcdf77fe640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcde003cd20 0x7fcde003f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.131+0000 7fcdf77fe640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcde003cd20 0x7fcde003f100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcde8006fd0 tx=0x7fcde8006df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:20.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.131+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fcde4000d40 con 0x7fcde003cd20 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 --> v2:172.21.15.100:6800/3986579393 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcde4002730 con 0x7fcde003cd20 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdf4ff9640 1 -- 172.21.15.100:0/4250424966 <== mgr.14150 v2:172.21.15.100:6800/3986579393 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fcde4002730 con 0x7fcde003cd20 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcde003cd20 msgr2=0x7fcde003f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcde003cd20 0x7fcde003f100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcde8006fd0 tx=0x7fcde8006df0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 msgr2=0x7fcdf810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 --2- 172.21.15.100:0/4250424966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf810f2c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcdec035040 tx=0x7fcdec0091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 shutdown_connections 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcde003cd20 0x7fcde003f100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 --2- 172.21.15.100:0/4250424966 >> v2:172.21.15.100:6800/2893280940 conn(0x7fcde00394c0 0x7fcde003b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 --2- 172.21.15.100:0/4250424966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcdf8100610 0x7fcdf810f2c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 >> 172.21.15.100:0/4250424966 conn(0x7fcdf80fbdd0 msgr2=0x7fcdf80fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:20.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 shutdown_connections 2024-09-10T18:24:20.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:20.147+0000 7fcdfe62a640 1 -- 172.21.15.100:0/4250424966 wait complete. 2024-09-10T18:24:20.829 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 9 is available 2024-09-10T18:24:20.829 INFO:teuthology.orchestra.run.smithi100.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:24:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: cephadm 2024-09-10T18:24:20.107364+0000 mgr.y (mgr.14150) 1 : cephadm [INF] [10/Sep/2024:18:24:20] ENGINE Bus STARTING 2024-09-10T18:24:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: audit 2024-09-10T18:24:20.136996+0000 mgr.y (mgr.14150) 2 : audit [DBG] from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: audit 2024-09-10T18:24:20.149647+0000 mgr.y (mgr.14150) 3 : audit [DBG] from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: cephadm 2024-09-10T18:24:20.233999+0000 mgr.y (mgr.14150) 4 : cephadm [INF] [10/Sep/2024:18:24:20] ENGINE Serving on https://172.21.15.100:7150 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: cephadm 2024-09-10T18:24:20.234304+0000 mgr.y (mgr.14150) 5 : cephadm [INF] [10/Sep/2024:18:24:20] ENGINE Bus STARTED 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: cephadm 2024-09-10T18:24:20.237645+0000 mgr.y (mgr.14150) 6 : cephadm [ERR] [10/Sep/2024:18:24:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: Traceback (most recent call last): 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: self._connections.run(self.expiration_interval) 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: self._run(expiration_interval) 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:24:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: s = self.context.wrap_socket( 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: return self.sslsocket_class._create( 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: self.do_handshake() 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: self._sslobj.do_handshake() 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:24:21.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:21 smithi100 bash[18597]: audit 2024-09-10T18:24:20.240748+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:22.512 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:22 smithi100 bash[18597]: cluster 2024-09-10T18:24:21.250656+0000 mon.a (mon.0) 83 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.074+0000 7f7498057640 1 Processor -- start 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.074+0000 7f7498057640 1 -- start start 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.074+0000 7f7498057640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f7490075b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.074+0000 7f7498057640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7490077960 con 0x7f7490077550 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.074+0000 7f7495df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f7490075b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.074+0000 7f7495df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f7490075b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39768/0 (socket says 172.21.15.100:39768) 2024-09-10T18:24:23.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.074+0000 7f7495df3640 1 -- 172.21.15.100:0/2526124447 learned_addr learned my addr 172.21.15.100:0/2526124447 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7495df3640 1 -- 172.21.15.100:0/2526124447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7490076060 con 0x7f7490077550 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7495df3640 1 --2- 172.21.15.100:0/2526124447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f7490075b20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7480009840 tx=0x7f74800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63255f2ac8b9d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7494df1640 1 -- 172.21.15.100:0/2526124447 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f748002add0 con 0x7f7490077550 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7494df1640 1 -- 172.21.15.100:0/2526124447 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f74800328b0 con 0x7f7490077550 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7494df1640 1 -- 172.21.15.100:0/2526124447 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7480031690 con 0x7f7490077550 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7498057640 1 -- 172.21.15.100:0/2526124447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 msgr2=0x7f7490075b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7498057640 1 --2- 172.21.15.100:0/2526124447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f7490075b20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7480009840 tx=0x7f74800091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7498057640 1 -- 172.21.15.100:0/2526124447 shutdown_connections 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7498057640 1 --2- 172.21.15.100:0/2526124447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f7490075b20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7498057640 1 -- 172.21.15.100:0/2526124447 >> 172.21.15.100:0/2526124447 conn(0x7f74900fbe10 msgr2=0x7f74900fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7498057640 1 -- 172.21.15.100:0/2526124447 shutdown_connections 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.078+0000 7f7498057640 1 -- 172.21.15.100:0/2526124447 wait complete. 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7498057640 1 Processor -- start 2024-09-10T18:24:23.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7498057640 1 -- start start 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7498057640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f74901158a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7498057640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7490115de0 con 0x7f7490077550 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7495df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f74901158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7495df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f74901158a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39776/0 (socket says 172.21.15.100:39776) 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7495df3640 1 -- 172.21.15.100:0/3035187183 learned_addr learned my addr 172.21.15.100:0/3035187183 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7495df3640 1 -- 172.21.15.100:0/3035187183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480009450 con 0x7f7490077550 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7495df3640 1 --2- 172.21.15.100:0/3035187183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f74901158a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7480006f00 tx=0x7f7480008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f74867fc640 1 -- 172.21.15.100:0/3035187183 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7480031c60 con 0x7f7490077550 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f74867fc640 1 -- 172.21.15.100:0/3035187183 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f7480031dc0 con 0x7f7490077550 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7490115fe0 con 0x7f7490077550 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f74867fc640 1 -- 172.21.15.100:0/3035187183 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f748003a540 con 0x7f7490077550 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7490116450 con 0x7f7490077550 2024-09-10T18:24:23.156 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.082+0000 7f74867fc640 1 -- 172.21.15.100:0/3035187183 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45470+0+0 (secure 0 0 0) 0x7f7480039070 con 0x7f7490077550 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.086+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749004b720 con 0x7f7490077550 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.086+0000 7f74867fc640 1 --2- 172.21.15.100:0/3035187183 >> v2:172.21.15.100:6800/3986579393 conn(0x7f747c0394c0 0x7f747c03b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.086+0000 7f74867fc640 1 -- 172.21.15.100:0/3035187183 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f748003b9a0 con 0x7f7490077550 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.086+0000 7f74955f2640 1 --2- 172.21.15.100:0/3035187183 >> v2:172.21.15.100:6800/3986579393 conn(0x7f747c0394c0 0x7f747c03b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.086+0000 7f74955f2640 1 --2- 172.21.15.100:0/3035187183 >> v2:172.21.15.100:6800/3986579393 conn(0x7f747c0394c0 0x7f747c03b970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f748c006fd0 tx=0x7f748c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.094+0000 7f74867fc640 1 -- 172.21.15.100:0/3035187183 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f748003a6f0 con 0x7f7490077550 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.434+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f7490116450 con 0x7f747c0394c0 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.478+0000 7f74867fc640 1 -- 172.21.15.100:0/3035187183 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f7490116450 con 0x7f747c0394c0 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.478+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 >> v2:172.21.15.100:6800/3986579393 conn(0x7f747c0394c0 msgr2=0x7f747c03b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.478+0000 7f7498057640 1 --2- 172.21.15.100:0/3035187183 >> v2:172.21.15.100:6800/3986579393 conn(0x7f747c0394c0 0x7f747c03b970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f748c006fd0 tx=0x7f748c006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.478+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 msgr2=0x7f74901158a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:23.157 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.478+0000 7f7498057640 1 --2- 172.21.15.100:0/3035187183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f74901158a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7480006f00 tx=0x7f7480008040 comp rx=0 tx=0).stop 2024-09-10T18:24:23.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.482+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 shutdown_connections 2024-09-10T18:24:23.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.482+0000 7f7498057640 1 --2- 172.21.15.100:0/3035187183 >> v2:172.21.15.100:6800/3986579393 conn(0x7f747c0394c0 0x7f747c03b970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:23.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.482+0000 7f7498057640 1 --2- 172.21.15.100:0/3035187183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7490077550 0x7f74901158a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:23.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.482+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 >> 172.21.15.100:0/3035187183 conn(0x7f74900fbe10 msgr2=0x7f749010cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:23.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.482+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 shutdown_connections 2024-09-10T18:24:23.158 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:22.482+0000 7f7498057640 1 -- 172.21.15.100:0/3035187183 wait complete. 2024-09-10T18:24:23.158 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial admin user... 2024-09-10T18:24:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:23 smithi100 bash[18597]: audit 2024-09-10T18:24:22.439759+0000 mgr.y (mgr.14150) 7 : audit [DBG] from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:24:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:23 smithi100 bash[18597]: audit 2024-09-10T18:24:22.476968+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:23 smithi100 bash[18597]: audit 2024-09-10T18:24:22.481978+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:25.171 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:24 smithi100 bash[18597]: audit 2024-09-10T18:24:24.721049+0000 mgr.y (mgr.14150) 8 : audit [DBG] from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:24:25.595 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$O.a5VwuiwW8p/dyqQU3pa.r1kS4nwWdSOKl7yYHuS96e8CHht5DYK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725992664, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:24:25.595 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.342+0000 7fd152bbd640 1 Processor -- start 2024-09-10T18:24:25.595 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 -- start start 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c1013c0 con 0x7fd14c100a70 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd151bbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd151bbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39792/0 (socket says 172.21.15.100:39792) 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd151bbb640 1 -- 172.21.15.100:0/3815869857 learned_addr learned my addr 172.21.15.100:0/3815869857 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd151bbb640 1 -- 172.21.15.100:0/3815869857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c101500 con 0x7fd14c100a70 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd151bbb640 1 --2- 172.21.15.100:0/3815869857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c100e80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd14800baf0 tx=0x7fd14800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac11084a7455024 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd150bb9640 1 -- 172.21.15.100:0/3815869857 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd148032640 con 0x7fd14c100a70 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd150bb9640 1 -- 172.21.15.100:0/3815869857 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd148032c10 con 0x7fd14c100a70 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd150bb9640 1 -- 172.21.15.100:0/3815869857 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd148031ae0 con 0x7fd14c100a70 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 -- 172.21.15.100:0/3815869857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 msgr2=0x7fd14c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 --2- 172.21.15.100:0/3815869857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c100e80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd14800baf0 tx=0x7fd14800b460 comp rx=0 tx=0).stop 2024-09-10T18:24:25.596 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 -- 172.21.15.100:0/3815869857 shutdown_connections 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 --2- 172.21.15.100:0/3815869857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c100e80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.346+0000 7fd152bbd640 1 -- 172.21.15.100:0/3815869857 >> 172.21.15.100:0/3815869857 conn(0x7fd14c0fa2c0 msgr2=0x7fd14c0fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 -- 172.21.15.100:0/3815869857 shutdown_connections 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 -- 172.21.15.100:0/3815869857 wait complete. 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 Processor -- start 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 -- start start 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c111330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c111870 con 0x7fd14c100a70 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd151bbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c111330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:25.597 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd151bbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c111330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39802/0 (socket says 172.21.15.100:39802) 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd151bbb640 1 -- 172.21.15.100:0/3571512735 learned_addr learned my addr 172.21.15.100:0/3571512735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd151bbb640 1 -- 172.21.15.100:0/3571512735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14800b700 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd151bbb640 1 --2- 172.21.15.100:0/3571512735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c111330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd14800b2c0 tx=0x7fd148008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd1427fc640 1 -- 172.21.15.100:0/3571512735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd148032920 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd1427fc640 1 -- 172.21.15.100:0/3571512735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd148032a80 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c111a70 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd1427fc640 1 -- 172.21.15.100:0/3571512735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1480453f0 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.350+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c111ee0 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.354+0000 7fd1427fc640 1 -- 172.21.15.100:0/3571512735 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45470+0+0 (secure 0 0 0) 0x7fd14803b420 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.354+0000 7fd1427fc640 1 --2- 172.21.15.100:0/3571512735 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd138039580 0x7fd13803ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.354+0000 7fd1427fc640 1 -- 172.21.15.100:0/3571512735 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fd14806d700 con 0x7fd14c100a70 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.354+0000 7fd1513ba640 1 --2- 172.21.15.100:0/3571512735 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd138039580 0x7fd13803ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:25.598 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.354+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c111ee0 con 0x7fd14c100a70 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.354+0000 7fd1513ba640 1 --2- 172.21.15.100:0/3571512735 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd138039580 0x7fd13803ba30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.362+0000 7fd1427fc640 1 -- 172.21.15.100:0/3571512735 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd148036020 con 0x7fd14c100a70 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.714+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd14c05ed70 con 0x7fd138039580 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.962+0000 7fd1427fc640 1 -- 172.21.15.100:0/3571512735 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd14c05ed70 con 0x7fd138039580 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd138039580 msgr2=0x7fd13803ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 --2- 172.21.15.100:0/3571512735 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd138039580 0x7fd13803ba30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 msgr2=0x7fd14c111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 --2- 172.21.15.100:0/3571512735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c111330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd14800b2c0 tx=0x7fd148008040 comp rx=0 tx=0).stop 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 shutdown_connections 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 --2- 172.21.15.100:0/3571512735 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd138039580 0x7fd13803ba30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 --2- 172.21.15.100:0/3571512735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd14c100a70 0x7fd14c111330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:25.599 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 >> 172.21.15.100:0/3571512735 conn(0x7fd14c0fa2c0 msgr2=0x7fd14c108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:25.600 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 shutdown_connections 2024-09-10T18:24:25.600 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:24.966+0000 7fd152bbd640 1 -- 172.21.15.100:0/3571512735 wait complete. 2024-09-10T18:24:25.600 INFO:teuthology.orchestra.run.smithi100.stdout:Fetching dashboard port number... 2024-09-10T18:24:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:25 smithi100 bash[18597]: audit 2024-09-10T18:24:24.927827+0000 mon.a (mon.0) 86 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:25 smithi100 bash[18597]: audit 2024-09-10T18:24:24.965821+0000 mon.a (mon.0) 87 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:25 smithi100 bash[18597]: audit 2024-09-10T18:24:25.510037+0000 mon.a (mon.0) 88 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:27.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:26 smithi100 bash[18597]: cluster 2024-09-10T18:24:25.971444+0000 mon.a (mon.0) 89 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-09-10T18:24:27.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:24:27.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6db84b640 1 Processor -- start 2024-09-10T18:24:27.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6db84b640 1 -- start start 2024-09-10T18:24:27.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6db84b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:27.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6db84b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4100f60 con 0x7fb6d4100610 2024-09-10T18:24:27.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6d95e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:27.829 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6d95e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d4100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39818/0 (socket says 172.21.15.100:39818) 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6d95e7640 1 -- 172.21.15.100:0/2585238469 learned_addr learned my addr 172.21.15.100:0/2585238469 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6d95e7640 1 -- 172.21.15.100:0/2585238469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d41010a0 con 0x7fb6d4100610 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.846+0000 7fb6d95e7640 1 --2- 172.21.15.100:0/2585238469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d4100a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009840 tx=0x7fb6d00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cb6dfa1c182307 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6cbfff640 1 -- 172.21.15.100:0/2585238469 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d002add0 con 0x7fb6d4100610 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6cbfff640 1 -- 172.21.15.100:0/2585238469 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fb6d00328b0 con 0x7fb6d4100610 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6cbfff640 1 -- 172.21.15.100:0/2585238469 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d0031690 con 0x7fb6d4100610 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 -- 172.21.15.100:0/2585238469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 msgr2=0x7fb6d4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 --2- 172.21.15.100:0/2585238469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d4100a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009840 tx=0x7fb6d00091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 -- 172.21.15.100:0/2585238469 shutdown_connections 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 --2- 172.21.15.100:0/2585238469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d4100a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 -- 172.21.15.100:0/2585238469 >> 172.21.15.100:0/2585238469 conn(0x7fb6d40fbe10 msgr2=0x7fb6d40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 -- 172.21.15.100:0/2585238469 shutdown_connections 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 -- 172.21.15.100:0/2585238469 wait complete. 2024-09-10T18:24:27.830 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 Processor -- start 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 -- start start 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d410f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6db84b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d410f740 con 0x7fb6d4100610 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6d95e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d410f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6d95e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d410f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39822/0 (socket says 172.21.15.100:39822) 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.850+0000 7fb6d95e7640 1 -- 172.21.15.100:0/2212986556 learned_addr learned my addr 172.21.15.100:0/2212986556 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6d95e7640 1 -- 172.21.15.100:0/2212986556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0009450 con 0x7fb6d4100610 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6d95e7640 1 --2- 172.21.15.100:0/2212986556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d410f200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0000c00 tx=0x7fb6d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6c9ffb640 1 -- 172.21.15.100:0/2212986556 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d0031c80 con 0x7fb6d4100610 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6c9ffb640 1 -- 172.21.15.100:0/2212986556 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fb6d0031de0 con 0x7fb6d4100610 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d410f940 con 0x7fb6d4100610 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6c9ffb640 1 -- 172.21.15.100:0/2212986556 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d003a540 con 0x7fb6d4100610 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d410fdb0 con 0x7fb6d4100610 2024-09-10T18:24:27.831 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d404b720 con 0x7fb6d4100610 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6c9ffb640 1 -- 172.21.15.100:0/2212986556 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7fb6d0039070 con 0x7fb6d4100610 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.854+0000 7fb6c9ffb640 1 --2- 172.21.15.100:0/2212986556 >> v2:172.21.15.100:6800/3986579393 conn(0x7fb6c0039510 0x7fb6c003b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.858+0000 7fb6c9ffb640 1 -- 172.21.15.100:0/2212986556 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fb6d006ca80 con 0x7fb6d4100610 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.858+0000 7fb6d8de6640 1 --2- 172.21.15.100:0/2212986556 >> v2:172.21.15.100:6800/3986579393 conn(0x7fb6c0039510 0x7fb6c003b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.858+0000 7fb6d8de6640 1 --2- 172.21.15.100:0/2212986556 >> v2:172.21.15.100:6800/3986579393 conn(0x7fb6c0039510 0x7fb6c003b9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006fd0 tx=0x7fb6c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:26.866+0000 7fb6c9ffb640 1 -- 172.21.15.100:0/2212986556 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb6d003b940 con 0x7fb6d4100610 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.190+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb6d4102380 con 0x7fb6d4100610 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.190+0000 7fb6c9ffb640 1 -- 172.21.15.100:0/2212986556 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb6d003f020 con 0x7fb6d4100610 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 >> v2:172.21.15.100:6800/3986579393 conn(0x7fb6c0039510 msgr2=0x7fb6c003b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 --2- 172.21.15.100:0/2212986556 >> v2:172.21.15.100:6800/3986579393 conn(0x7fb6c0039510 0x7fb6c003b9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4006fd0 tx=0x7fb6c4006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 msgr2=0x7fb6d410f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 --2- 172.21.15.100:0/2212986556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d410f200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0000c00 tx=0x7fb6d0008040 comp rx=0 tx=0).stop 2024-09-10T18:24:27.832 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 shutdown_connections 2024-09-10T18:24:27.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 --2- 172.21.15.100:0/2212986556 >> v2:172.21.15.100:6800/3986579393 conn(0x7fb6c0039510 0x7fb6c003b9c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:27.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 --2- 172.21.15.100:0/2212986556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6d4100610 0x7fb6d410f200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:27.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 >> 172.21.15.100:0/2212986556 conn(0x7fb6d40fbe10 msgr2=0x7fb6d40fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:27.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 shutdown_connections 2024-09-10T18:24:27.833 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:27.194+0000 7fb6db84b640 1 -- 172.21.15.100:0/2212986556 wait complete. 2024-09-10T18:24:27.833 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld does not appear to be present 2024-09-10T18:24:27.833 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:24:27.834 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph Dashboard is now available at: 2024-09-10T18:24:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: URL: https://smithi100.front.sepia.ceph.com:8443/ 2024-09-10T18:24:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: User: admin 2024-09-10T18:24:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: Password: 1pqw8adhtg 2024-09-10T18:24:27.834 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:27.834 INFO:teuthology.orchestra.run.smithi100.stdout:Saving cluster configuration to /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config directory 2024-09-10T18:24:27.835 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:24:28.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:27 smithi100 bash[18597]: audit 2024-09-10T18:24:27.196518+0000 mon.a (mon.0) 90 : audit [DBG] from='client.? 172.21.15.100:0/2212986556' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.014+0000 7f3d52e0d640 1 Processor -- start 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.014+0000 7f3d52e0d640 1 -- start start 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.014+0000 7f3d52e0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.014+0000 7f3d52e0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c1031e0 con 0x7f3d4c102890 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.014+0000 7f3d50ba9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.014+0000 7f3d50ba9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39830/0 (socket says 172.21.15.100:39830) 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.014+0000 7f3d50ba9640 1 -- 172.21.15.100:0/110742425 learned_addr learned my addr 172.21.15.100:0/110742425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d50ba9640 1 -- 172.21.15.100:0/110742425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c103320 con 0x7f3d4c102890 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d50ba9640 1 --2- 172.21.15.100:0/110742425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c102ca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009e50 tx=0x7f3d44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0ee4771138a575 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d4b7fe640 1 -- 172.21.15.100:0/110742425 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d44031400 con 0x7f3d4c102890 2024-09-10T18:24:30.019 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d4b7fe640 1 -- 172.21.15.100:0/110742425 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f3d4402d050 con 0x7f3d4c102890 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d4b7fe640 1 -- 172.21.15.100:0/110742425 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d44007690 con 0x7f3d4c102890 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 -- 172.21.15.100:0/110742425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 msgr2=0x7f3d4c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 --2- 172.21.15.100:0/110742425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c102ca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009e50 tx=0x7f3d44009840 comp rx=0 tx=0).stop 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 -- 172.21.15.100:0/110742425 shutdown_connections 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 --2- 172.21.15.100:0/110742425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c102ca0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 -- 172.21.15.100:0/110742425 >> 172.21.15.100:0/110742425 conn(0x7f3d4c075f80 msgr2=0x7f3d4c076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 -- 172.21.15.100:0/110742425 shutdown_connections 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 -- 172.21.15.100:0/110742425 wait complete. 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 Processor -- start 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.018+0000 7f3d52e0d640 1 -- start start 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d52e0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d52e0d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c111a40 con 0x7f3d4c102890 2024-09-10T18:24:30.020 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d50ba9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d50ba9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39836/0 (socket says 172.21.15.100:39836) 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d50ba9640 1 -- 172.21.15.100:0/2789167028 learned_addr learned my addr 172.21.15.100:0/2789167028 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d50ba9640 1 -- 172.21.15.100:0/2789167028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d44009ae0 con 0x7f3d4c102890 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d50ba9640 1 --2- 172.21.15.100:0/2789167028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c111500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3d44003400 tx=0x7f3d44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d497fa640 1 -- 172.21.15.100:0/2789167028 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d440088e0 con 0x7f3d4c102890 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d497fa640 1 -- 172.21.15.100:0/2789167028 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f3d44008eb0 con 0x7f3d4c102890 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d497fa640 1 -- 172.21.15.100:0/2789167028 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d4403bbc0 con 0x7f3d4c102890 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c111c40 con 0x7f3d4c102890 2024-09-10T18:24:30.021 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c1120b0 con 0x7f3d4c102890 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d497fa640 1 -- 172.21.15.100:0/2789167028 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f3d44038050 con 0x7f3d4c102890 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c04b720 con 0x7f3d4c102890 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d497fa640 1 --2- 172.21.15.100:0/2789167028 >> v2:172.21.15.100:6800/3986579393 conn(0x7f3d38039560 0x7f3d3803ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.022+0000 7f3d497fa640 1 -- 172.21.15.100:0/2789167028 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f3d4406c5f0 con 0x7f3d4c102890 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.026+0000 7f3d4bfff640 1 --2- 172.21.15.100:0/2789167028 >> v2:172.21.15.100:6800/3986579393 conn(0x7f3d38039560 0x7f3d3803ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.026+0000 7f3d4bfff640 1 --2- 172.21.15.100:0/2789167028 >> v2:172.21.15.100:6800/3986579393 conn(0x7f3d38039560 0x7f3d3803ba10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3d34006fd0 tx=0x7f3d34006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.030+0000 7f3d497fa640 1 -- 172.21.15.100:0/2789167028 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3d44039300 con 0x7f3d4c102890 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.358+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f3d4c1045b0 con 0x7f3d4c102890 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d497fa640 1 -- 172.21.15.100:0/2789167028 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f3d440410c0 con 0x7f3d4c102890 2024-09-10T18:24:30.022 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 >> v2:172.21.15.100:6800/3986579393 conn(0x7f3d38039560 msgr2=0x7f3d3803ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 --2- 172.21.15.100:0/2789167028 >> v2:172.21.15.100:6800/3986579393 conn(0x7f3d38039560 0x7f3d3803ba10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3d34006fd0 tx=0x7f3d34006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 msgr2=0x7f3d4c111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 --2- 172.21.15.100:0/2789167028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c111500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3d44003400 tx=0x7f3d44009840 comp rx=0 tx=0).stop 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 shutdown_connections 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 --2- 172.21.15.100:0/2789167028 >> v2:172.21.15.100:6800/3986579393 conn(0x7f3d38039560 0x7f3d3803ba10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 --2- 172.21.15.100:0/2789167028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d4c102890 0x7f3d4c111500 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 >> 172.21.15.100:0/2789167028 conn(0x7f3d4c075f80 msgr2=0x7f3d4c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 shutdown_connections 2024-09-10T18:24:30.023 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:29.362+0000 7f3d52e0d640 1 -- 172.21.15.100:0/2789167028 wait complete. 2024-09-10T18:24:33.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.618+0000 7f18ea4f5640 1 Processor -- start 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18ea4f5640 1 -- start start 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18ea4f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e41055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18ea4f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4100fc0 con 0x7f18e4074c00 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e41055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e41055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54790/0 (socket says 172.21.15.100:54790) 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e3fff640 1 -- 172.21.15.100:0/184366798 learned_addr learned my addr 172.21.15.100:0/184366798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e3fff640 1 -- 172.21.15.100:0/184366798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4105b10 con 0x7f18e4074c00 2024-09-10T18:24:33.146 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e3fff640 1 --2- 172.21.15.100:0/184366798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e41055d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009840 tx=0x7f18d40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fb5c7ddafe4328 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:33.150 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e2ffd640 1 -- 172.21.15.100:0/184366798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d402add0 con 0x7f18e4074c00 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e2ffd640 1 -- 172.21.15.100:0/184366798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f18d40328b0 con 0x7f18e4074c00 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.622+0000 7f18e2ffd640 1 -- 172.21.15.100:0/184366798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d4031690 con 0x7f18e4074c00 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- 172.21.15.100:0/184366798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 msgr2=0x7f18e41055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 --2- 172.21.15.100:0/184366798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e41055d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009840 tx=0x7f18d40091d0 comp rx=0 tx=0).stop 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- 172.21.15.100:0/184366798 shutdown_connections 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 --2- 172.21.15.100:0/184366798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e41055d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- 172.21.15.100:0/184366798 >> 172.21.15.100:0/184366798 conn(0x7f18e40fabc0 msgr2=0x7f18e40fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- 172.21.15.100:0/184366798 shutdown_connections 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- 172.21.15.100:0/184366798 wait complete. 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 Processor -- start 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- start start 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e4111510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4111a50 con 0x7f18e4074c00 2024-09-10T18:24:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e4111510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e4111510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54792/0 (socket says 172.21.15.100:54792) 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18e3fff640 1 -- 172.21.15.100:0/2189103195 learned_addr learned my addr 172.21.15.100:0/2189103195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18e3fff640 1 -- 172.21.15.100:0/2189103195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4009450 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18e3fff640 1 --2- 172.21.15.100:0/2189103195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e4111510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000c00 tx=0x7f18d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18e0ff9640 1 -- 172.21.15.100:0/2189103195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d4031c50 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18e0ff9640 1 -- 172.21.15.100:0/2189103195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f18d4031db0 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.626+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e4111c50 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e41120c0 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18e0ff9640 1 -- 172.21.15.100:0/2189103195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d403a3a0 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18e0ff9640 1 -- 172.21.15.100:0/2189103195 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f18d4039070 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18e0ff9640 1 --2- 172.21.15.100:0/2189103195 >> v2:172.21.15.100:6800/3986579393 conn(0x7f18d0039560 0x7f18d003ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18e0ff9640 1 -- 172.21.15.100:0/2189103195 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f18d406c9c0 con 0x7f18e4074c00 2024-09-10T18:24:33.152 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18e37fe640 1 --2- 172.21.15.100:0/2189103195 >> v2:172.21.15.100:6800/3986579393 conn(0x7f18d0039560 0x7f18d003ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18e37fe640 1 --2- 172.21.15.100:0/2189103195 >> v2:172.21.15.100:6800/3986579393 conn(0x7f18d0039560 0x7f18d003ba10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006fd0 tx=0x7f18cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.630+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c4005040 con 0x7f18e4074c00 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:31.638+0000 7f18e0ff9640 1 -- 172.21.15.100:0/2189103195 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f18d403f080 con 0x7f18e4074c00 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.158+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f18c4005870 con 0x7f18e4074c00 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.158+0000 7f18e0ff9640 1 -- 172.21.15.100:0/2189103195 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f18d4037070 con 0x7f18e4074c00 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 >> v2:172.21.15.100:6800/3986579393 conn(0x7f18d0039560 msgr2=0x7f18d003ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 --2- 172.21.15.100:0/2189103195 >> v2:172.21.15.100:6800/3986579393 conn(0x7f18d0039560 0x7f18d003ba10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006fd0 tx=0x7f18cc006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 msgr2=0x7f18e4111510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 --2- 172.21.15.100:0/2189103195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e4111510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000c00 tx=0x7f18d4008040 comp rx=0 tx=0).stop 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 shutdown_connections 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 --2- 172.21.15.100:0/2189103195 >> v2:172.21.15.100:6800/3986579393 conn(0x7f18d0039560 0x7f18d003ba10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:33.153 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 --2- 172.21.15.100:0/2189103195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18e4074c00 0x7f18e4111510 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:33.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 >> 172.21.15.100:0/2189103195 conn(0x7f18e40fabc0 msgr2=0x7f18e40fb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:33.154 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 shutdown_connections 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-09-10T18:24:32.162+0000 7f18ea4f5640 1 -- 172.21.15.100:0/2189103195 wait complete. 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: ceph telemetry on 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout:For more information see: 2024-09-10T18:24:33.155 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.156 INFO:teuthology.orchestra.run.smithi100.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:24:33.156 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:24:33.156 INFO:teuthology.orchestra.run.smithi100.stdout:Bootstrap complete. 2024-09-10T18:24:33.186 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:24:33.186 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:24:33.186 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:24:33.193 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:24:33.193 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:24:33.193 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:24:33.241 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:24:33.241 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:24:33.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:24:33.252 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:24:33.253 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:24:33.253 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:24:33.258 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:24:33.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDToza/hxKAXIIBofZJbo+7MnC9mvD43Cr+oSLJV/7UaF5vYpyC1mumsSIvTzUZ5aK40GVZP7isbj7LG6MqnGlXsCGD8QqEPkNvVDE6Tx9D8cD5eqhCFLDL6osaxD2sJEo4cWQEYjh6/Pjq9JRUUcpda9ooxmO5I5vDQBKGJqHhMJQJW/IeGA93mcnYaMI45bYG9dW3qcZUI1rWwuFRNmg6XgRCcS3kz3OjtzF3EJU9s5kqObujt/XRrs567Qk581U5g+hq4rUqSEogDyTtEvO5xJPfZOgeuuG9Q9JFdmECCUVgehJjVogJKckC5a/+qZ/wfMtQlOI+ADe5WV1Th1enFGjqSWk32ZijHckw3lKjEo3QSzBPNqOp7/uSmMAtmbReqCygoCaVMob1sO1qnNg6p4ZZfGLrYSXxNFToQs5Oc8glYUohTZwxHmtMCYC7UwVwE24qitVxzWkOGvl++gr2JyxAvBotOh1BL6t2w2BWKlW8DGgSeobMficvWBsAaWs= ceph-505ebbde-6fa1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:24:33.280 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDToza/hxKAXIIBofZJbo+7MnC9mvD43Cr+oSLJV/7UaF5vYpyC1mumsSIvTzUZ5aK40GVZP7isbj7LG6MqnGlXsCGD8QqEPkNvVDE6Tx9D8cD5eqhCFLDL6osaxD2sJEo4cWQEYjh6/Pjq9JRUUcpda9ooxmO5I5vDQBKGJqHhMJQJW/IeGA93mcnYaMI45bYG9dW3qcZUI1rWwuFRNmg6XgRCcS3kz3OjtzF3EJU9s5kqObujt/XRrs567Qk581U5g+hq4rUqSEogDyTtEvO5xJPfZOgeuuG9Q9JFdmECCUVgehJjVogJKckC5a/+qZ/wfMtQlOI+ADe5WV1Th1enFGjqSWk32ZijHckw3lKjEo3QSzBPNqOp7/uSmMAtmbReqCygoCaVMob1sO1qnNg6p4ZZfGLrYSXxNFToQs5Oc8glYUohTZwxHmtMCYC7UwVwE24qitVxzWkOGvl++gr2JyxAvBotOh1BL6t2w2BWKlW8DGgSeobMficvWBsAaWs= ceph-505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:24:33.291 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDToza/hxKAXIIBofZJbo+7MnC9mvD43Cr+oSLJV/7UaF5vYpyC1mumsSIvTzUZ5aK40GVZP7isbj7LG6MqnGlXsCGD8QqEPkNvVDE6Tx9D8cD5eqhCFLDL6osaxD2sJEo4cWQEYjh6/Pjq9JRUUcpda9ooxmO5I5vDQBKGJqHhMJQJW/IeGA93mcnYaMI45bYG9dW3qcZUI1rWwuFRNmg6XgRCcS3kz3OjtzF3EJU9s5kqObujt/XRrs567Qk581U5g+hq4rUqSEogDyTtEvO5xJPfZOgeuuG9Q9JFdmECCUVgehJjVogJKckC5a/+qZ/wfMtQlOI+ADe5WV1Th1enFGjqSWk32ZijHckw3lKjEo3QSzBPNqOp7/uSmMAtmbReqCygoCaVMob1sO1qnNg6p4ZZfGLrYSXxNFToQs5Oc8glYUohTZwxHmtMCYC7UwVwE24qitVxzWkOGvl++gr2JyxAvBotOh1BL6t2w2BWKlW8DGgSeobMficvWBsAaWs= ceph-505ebbde-6fa1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:24:33.320 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDToza/hxKAXIIBofZJbo+7MnC9mvD43Cr+oSLJV/7UaF5vYpyC1mumsSIvTzUZ5aK40GVZP7isbj7LG6MqnGlXsCGD8QqEPkNvVDE6Tx9D8cD5eqhCFLDL6osaxD2sJEo4cWQEYjh6/Pjq9JRUUcpda9ooxmO5I5vDQBKGJqHhMJQJW/IeGA93mcnYaMI45bYG9dW3qcZUI1rWwuFRNmg6XgRCcS3kz3OjtzF3EJU9s5kqObujt/XRrs567Qk581U5g+hq4rUqSEogDyTtEvO5xJPfZOgeuuG9Q9JFdmECCUVgehJjVogJKckC5a/+qZ/wfMtQlOI+ADe5WV1Th1enFGjqSWk32ZijHckw3lKjEo3QSzBPNqOp7/uSmMAtmbReqCygoCaVMob1sO1qnNg6p4ZZfGLrYSXxNFToQs5Oc8glYUohTZwxHmtMCYC7UwVwE24qitVxzWkOGvl++gr2JyxAvBotOh1BL6t2w2BWKlW8DGgSeobMficvWBsAaWs= ceph-505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:24:33.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:24:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:33 smithi100 bash[18597]: audit 2024-09-10T18:24:32.164654+0000 mon.a (mon.0) 91 : audit [INF] from='client.? 172.21.15.100:0/2189103195' entity='client.admin' 2024-09-10T18:24:38.166 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:24:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:40 smithi100 bash[18597]: cluster 2024-09-10T18:24:39.138332+0000 mgr.y (mgr.14150) 9 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- 172.21.15.100:0/468401678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 msgr2=0x7f84bc0fb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 --2- 172.21.15.100:0/468401678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0fb6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84ac004850 tx=0x7f84ac009da0 comp rx=0 tx=0).stop 2024-09-10T18:24:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- 172.21.15.100:0/468401678 shutdown_connections 2024-09-10T18:24:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 --2- 172.21.15.100:0/468401678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0fb6f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- 172.21.15.100:0/468401678 >> 172.21.15.100:0/468401678 conn(0x7f84bc0f6ac0 msgr2=0x7f84bc0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:41.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- 172.21.15.100:0/468401678 shutdown_connections 2024-09-10T18:24:41.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- 172.21.15.100:0/468401678 wait complete. 2024-09-10T18:24:41.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 Processor -- start 2024-09-10T18:24:41.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- start start 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc0f8f20 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c1240640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c1240640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55780/0 (socket says 172.21.15.100:55780) 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c1240640 1 -- 172.21.15.100:0/1047551919 learned_addr learned my addr 172.21.15.100:0/1047551919 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c1240640 1 -- 172.21.15.100:0/1047551919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac005040 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c1240640 1 --2- 172.21.15.100:0/1047551919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0f89e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0048b0 tx=0x7f84ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84b9ffb640 1 -- 172.21.15.100:0/1047551919 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84ac038070 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc0f6fb0 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84b9ffb640 1 -- 172.21.15.100:0/1047551919 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f84ac0098a0 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.638+0000 7f84b9ffb640 1 -- 172.21.15.100:0/1047551919 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84ac0335c0 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.642+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc0f7420 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.642+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc100ca0 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.642+0000 7f84b9ffb640 1 -- 172.21.15.100:0/1047551919 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f84ac030020 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.642+0000 7f84b9ffb640 1 --2- 172.21.15.100:0/1047551919 >> v2:172.21.15.100:6800/3986579393 conn(0x7f84a40394d0 0x7f84a403b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.642+0000 7f84b9ffb640 1 -- 172.21.15.100:0/1047551919 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f84ac06c4f0 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.642+0000 7f84c0a3f640 1 --2- 172.21.15.100:0/1047551919 >> v2:172.21.15.100:6800/3986579393 conn(0x7f84a40394d0 0x7f84a403b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:41.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.642+0000 7f84c0a3f640 1 --2- 172.21.15.100:0/1047551919 >> v2:172.21.15.100:6800/3986579393 conn(0x7f84a40394d0 0x7f84a403b980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f84b0006fd0 tx=0x7f84b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.650+0000 7f84b9ffb640 1 -- 172.21.15.100:0/1047551919 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f84ac033830 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:41 smithi100 bash[18597]: audit 2024-09-10T18:24:40.562272+0000 mon.a (mon.0) 92 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:41 smithi100 bash[18597]: audit 2024-09-10T18:24:40.568465+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:41 smithi100 bash[18597]: audit 2024-09-10T18:24:40.569602+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:24:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:41 smithi100 bash[18597]: audit 2024-09-10T18:24:40.574670+0000 mon.a (mon.0) 95 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:41 smithi100 bash[18597]: audit 2024-09-10T18:24:40.597348+0000 mon.a (mon.0) 96 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:41 smithi100 bash[18597]: audit 2024-09-10T18:24:40.605782+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.822+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f84bc05ed70 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84b9ffb640 1 -- 172.21.15.100:0/1047551919 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f84ac03c3c0 con 0x7f84bc0fb2e0 2024-09-10T18:24:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 >> v2:172.21.15.100:6800/3986579393 conn(0x7f84a40394d0 msgr2=0x7f84a403b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 --2- 172.21.15.100:0/1047551919 >> v2:172.21.15.100:6800/3986579393 conn(0x7f84a40394d0 0x7f84a403b980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f84b0006fd0 tx=0x7f84b0006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 msgr2=0x7f84bc0f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 --2- 172.21.15.100:0/1047551919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0f89e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0048b0 tx=0x7f84ac009da0 comp rx=0 tx=0).stop 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 shutdown_connections 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 --2- 172.21.15.100:0/1047551919 >> v2:172.21.15.100:6800/3986579393 conn(0x7f84a40394d0 0x7f84a403b980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 --2- 172.21.15.100:0/1047551919 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84bc0fb2e0 0x7f84bc0f89e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 >> 172.21.15.100:0/1047551919 conn(0x7f84bc0f6ac0 msgr2=0x7f84bc064990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 shutdown_connections 2024-09-10T18:24:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:41.826+0000 7f84c34a4640 1 -- 172.21.15.100:0/1047551919 wait complete. 2024-09-10T18:24:42.507 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:24:42.508 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:24:43.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:42 smithi100 bash[18597]: cluster 2024-09-10T18:24:41.138773+0000 mgr.y (mgr.14150) 10 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:43.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:42 smithi100 bash[18597]: audit 2024-09-10T18:24:41.829558+0000 mon.a (mon.0) 98 : audit [INF] from='client.? 172.21.15.100:0/1047551919' entity='client.admin' 2024-09-10T18:24:43.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:42 smithi100 bash[18597]: audit 2024-09-10T18:24:41.850730+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:43.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:42 smithi100 bash[18597]: audit 2024-09-10T18:24:41.856549+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:44 smithi100 bash[18597]: cluster 2024-09-10T18:24:43.139236+0000 mgr.y (mgr.14150) 11 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:47.228 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:24:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:46 smithi100 bash[18597]: cluster 2024-09-10T18:24:45.139703+0000 mgr.y (mgr.14150) 12 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:49.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:48 smithi100 bash[18597]: cluster 2024-09-10T18:24:47.140129+0000 mgr.y (mgr.14150) 13 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:50.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 -- 172.21.15.100:0/2423350331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 msgr2=0x7fea840ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:50.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 --2- 172.21.15.100:0/2423350331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea840ff090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fea78004850 tx=0x7fea78009da0 comp rx=0 tx=0).stop 2024-09-10T18:24:50.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 -- 172.21.15.100:0/2423350331 shutdown_connections 2024-09-10T18:24:50.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 --2- 172.21.15.100:0/2423350331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea840ff090 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 -- 172.21.15.100:0/2423350331 >> 172.21.15.100:0/2423350331 conn(0x7fea840f6ac0 msgr2=0x7fea840f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 -- 172.21.15.100:0/2423350331 shutdown_connections 2024-09-10T18:24:50.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 -- 172.21.15.100:0/2423350331 wait complete. 2024-09-10T18:24:50.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 Processor -- start 2024-09-10T18:24:50.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 -- start start 2024-09-10T18:24:50.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea8410bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:50.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea8b0d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8410c430 con 0x7fea840655b0 2024-09-10T18:24:50.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea88e71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea8410bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:50.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea88e71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea8410bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43974/0 (socket says 172.21.15.100:43974) 2024-09-10T18:24:50.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea88e71640 1 -- 172.21.15.100:0/146503296 learned_addr learned my addr 172.21.15.100:0/146503296 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:50.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea88e71640 1 -- 172.21.15.100:0/146503296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea78005040 con 0x7fea840655b0 2024-09-10T18:24:50.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.206+0000 7fea88e71640 1 --2- 172.21.15.100:0/146503296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea8410bef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea7802a6d0 tx=0x7fea78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:50.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea817fa640 1 -- 172.21.15.100:0/146503296 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea78038070 con 0x7fea840655b0 2024-09-10T18:24:50.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea817fa640 1 -- 172.21.15.100:0/146503296 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fea780098a0 con 0x7fea840655b0 2024-09-10T18:24:50.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8410c630 con 0x7fea840655b0 2024-09-10T18:24:50.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea817fa640 1 -- 172.21.15.100:0/146503296 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea7803c5b0 con 0x7fea840655b0 2024-09-10T18:24:50.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8410d080 con 0x7fea840655b0 2024-09-10T18:24:50.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea817fa640 1 -- 172.21.15.100:0/146503296 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7fea78030020 con 0x7fea840655b0 2024-09-10T18:24:50.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea84101da0 con 0x7fea840655b0 2024-09-10T18:24:50.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea817fa640 1 --2- 172.21.15.100:0/146503296 >> v2:172.21.15.100:6800/3986579393 conn(0x7fea6c039560 0x7fea6c03ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:50.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea817fa640 1 -- 172.21.15.100:0/146503296 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fea7806c280 con 0x7fea840655b0 2024-09-10T18:24:50.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea83fff640 1 --2- 172.21.15.100:0/146503296 >> v2:172.21.15.100:6800/3986579393 conn(0x7fea6c039560 0x7fea6c03ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:50.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.210+0000 7fea83fff640 1 --2- 172.21.15.100:0/146503296 >> v2:172.21.15.100:6800/3986579393 conn(0x7fea6c039560 0x7fea6c03ba10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea74006fd0 tx=0x7fea74006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:50.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.218+0000 7fea817fa640 1 -- 172.21.15.100:0/146503296 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fea7803c710 con 0x7fea840655b0 2024-09-10T18:24:50.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.398+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fea841015f0 con 0x7fea6c039560 2024-09-10T18:24:50.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.402+0000 7fea817fa640 1 -- 172.21.15.100:0/146503296 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fea841015f0 con 0x7fea6c039560 2024-09-10T18:24:50.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 >> v2:172.21.15.100:6800/3986579393 conn(0x7fea6c039560 msgr2=0x7fea6c03ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 --2- 172.21.15.100:0/146503296 >> v2:172.21.15.100:6800/3986579393 conn(0x7fea6c039560 0x7fea6c03ba10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea74006fd0 tx=0x7fea74006db0 comp rx=0 tx=0).stop 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 msgr2=0x7fea8410bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 --2- 172.21.15.100:0/146503296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea8410bef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea7802a6d0 tx=0x7fea78009da0 comp rx=0 tx=0).stop 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 shutdown_connections 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 --2- 172.21.15.100:0/146503296 >> v2:172.21.15.100:6800/3986579393 conn(0x7fea6c039560 0x7fea6c03ba10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 --2- 172.21.15.100:0/146503296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea840655b0 0x7fea8410bef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 >> 172.21.15.100:0/146503296 conn(0x7fea840f6ac0 msgr2=0x7fea840f7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 shutdown_connections 2024-09-10T18:24:50.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:50.410+0000 7fea8b0d5640 1 -- 172.21.15.100:0/146503296 wait complete. 2024-09-10T18:24:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:50 smithi100 bash[18597]: cluster 2024-09-10T18:24:49.140555+0000 mgr.y (mgr.14150) 14 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:50 smithi100 bash[18597]: audit 2024-09-10T18:24:50.408512+0000 mon.a (mon.0) 101 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:50 smithi100 bash[18597]: audit 2024-09-10T18:24:50.409787+0000 mon.a (mon.0) 102 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:24:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:50 smithi100 bash[18597]: audit 2024-09-10T18:24:50.412365+0000 mon.a (mon.0) 103 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:24:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:50 smithi100 bash[18597]: audit 2024-09-10T18:24:50.413811+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:24:51.117 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-09-10T18:24:51.118 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:24:51.118 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:24:51.125 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:24:51.125 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:24:51.132 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-09-10T18:24:51.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch host add smithi151 2024-09-10T18:24:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: audit 2024-09-10T18:24:50.403788+0000 mgr.y (mgr.14150) 15 : audit [DBG] from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:24:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: cephadm 2024-09-10T18:24:50.415507+0000 mgr.y (mgr.14150) 16 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:24:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: cephadm 2024-09-10T18:24:50.526257+0000 mgr.y (mgr.14150) 17 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:24:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: cephadm 2024-09-10T18:24:50.636936+0000 mgr.y (mgr.14150) 18 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:24:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: cephadm 2024-09-10T18:24:50.785980+0000 mgr.y (mgr.14150) 19 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:24:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: audit 2024-09-10T18:24:50.907027+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:52.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: audit 2024-09-10T18:24:50.910308+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:52.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:51 smithi100 bash[18597]: audit 2024-09-10T18:24:50.918972+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:24:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:52 smithi100 bash[18597]: cluster 2024-09-10T18:24:51.141029+0000 mgr.y (mgr.14150) 20 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:54 smithi100 bash[18597]: cluster 2024-09-10T18:24:53.141485+0000 mgr.y (mgr.14150) 21 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:55.853 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:24:57.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:56 smithi100 bash[18597]: cluster 2024-09-10T18:24:55.141841+0000 mgr.y (mgr.14150) 22 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:58.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.946+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/29810141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 msgr2=0x7fe7f01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:24:58.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.946+0000 7fe7f7bc8640 1 --2- 172.21.15.100:0/29810141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f01025d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4004850 tx=0x7fe7e4009da0 comp rx=0 tx=0).stop 2024-09-10T18:24:58.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.946+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/29810141 shutdown_connections 2024-09-10T18:24:58.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.946+0000 7fe7f7bc8640 1 --2- 172.21.15.100:0/29810141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f01025d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:24:58.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/29810141 >> 172.21.15.100:0/29810141 conn(0x7fe7f00f7dd0 msgr2=0x7fe7f00fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:24:58.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/29810141 shutdown_connections 2024-09-10T18:24:58.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/29810141 wait complete. 2024-09-10T18:24:58.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 Processor -- start 2024-09-10T18:24:58.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 -- start start 2024-09-10T18:24:58.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f0111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:58.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f01121e0 con 0x7fe7f0102200 2024-09-10T18:24:58.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f5964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f0111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:58.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f5964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f0111ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43984/0 (socket says 172.21.15.100:43984) 2024-09-10T18:24:58.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f5964640 1 -- 172.21.15.100:0/943559333 learned_addr learned my addr 172.21.15.100:0/943559333 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:24:58.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f5964640 1 -- 172.21.15.100:0/943559333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4005040 con 0x7fe7f0102200 2024-09-10T18:24:58.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f5964640 1 --2- 172.21.15.100:0/943559333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f0111ca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4003080 tx=0x7fe7e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:58.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7e27fc640 1 -- 172.21.15.100:0/943559333 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7e4038070 con 0x7fe7f0102200 2024-09-10T18:24:58.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7e27fc640 1 -- 172.21.15.100:0/943559333 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fe7e40093b0 con 0x7fe7f0102200 2024-09-10T18:24:58.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7e27fc640 1 -- 172.21.15.100:0/943559333 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7e403c410 con 0x7fe7f0102200 2024-09-10T18:24:58.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f01123e0 con 0x7fe7f0102200 2024-09-10T18:24:58.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.950+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f0112850 con 0x7fe7f0102200 2024-09-10T18:24:58.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.954+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f00fd970 con 0x7fe7f0102200 2024-09-10T18:24:58.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.954+0000 7fe7e27fc640 1 -- 172.21.15.100:0/943559333 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7fe7e4030020 con 0x7fe7f0102200 2024-09-10T18:24:58.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.954+0000 7fe7e27fc640 1 --2- 172.21.15.100:0/943559333 >> v2:172.21.15.100:6800/3986579393 conn(0x7fe7dc039560 0x7fe7dc03ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:24:58.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.954+0000 7fe7e27fc640 1 -- 172.21.15.100:0/943559333 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe7e406bc30 con 0x7fe7f0102200 2024-09-10T18:24:58.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.954+0000 7fe7f5163640 1 --2- 172.21.15.100:0/943559333 >> v2:172.21.15.100:6800/3986579393 conn(0x7fe7dc039560 0x7fe7dc03ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:24:58.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.954+0000 7fe7f5163640 1 --2- 172.21.15.100:0/943559333 >> v2:172.21.15.100:6800/3986579393 conn(0x7fe7dc039560 0x7fe7dc03ba10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800ad90 tx=0x7fe7e8009410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:24:58.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:58.962+0000 7fe7e27fc640 1 -- 172.21.15.100:0/943559333 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe7e403bdc0 con 0x7fe7f0102200 2024-09-10T18:24:59.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:24:58 smithi100 bash[18597]: cluster 2024-09-10T18:24:57.142257+0000 mgr.y (mgr.14150) 23 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:24:59.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:24:59.146+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7f01059f0 con 0x7fe7dc039560 2024-09-10T18:25:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:00 smithi100 bash[18597]: cluster 2024-09-10T18:24:59.142654+0000 mgr.y (mgr.14150) 24 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:00 smithi100 bash[18597]: audit 2024-09-10T18:24:59.151001+0000 mgr.y (mgr.14150) 25 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:25:02.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:01 smithi100 bash[18597]: cephadm 2024-09-10T18:25:00.398364+0000 mgr.y (mgr.14150) 26 : cephadm [INF] Deploying cephadm binary to smithi151 2024-09-10T18:25:03.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.001+0000 7fe7e27fc640 1 -- 172.21.15.100:0/943559333 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe7f01059f0 con 0x7fe7dc039560 2024-09-10T18:25:03.006 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-09-10T18:25:03.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 >> v2:172.21.15.100:6800/3986579393 conn(0x7fe7dc039560 msgr2=0x7fe7dc03ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:03.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 --2- 172.21.15.100:0/943559333 >> v2:172.21.15.100:6800/3986579393 conn(0x7fe7dc039560 0x7fe7dc03ba10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800ad90 tx=0x7fe7e8009410 comp rx=0 tx=0).stop 2024-09-10T18:25:03.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 msgr2=0x7fe7f0111ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:03.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 --2- 172.21.15.100:0/943559333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f0111ca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4003080 tx=0x7fe7e4009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:03.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 shutdown_connections 2024-09-10T18:25:03.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 --2- 172.21.15.100:0/943559333 >> v2:172.21.15.100:6800/3986579393 conn(0x7fe7dc039560 0x7fe7dc03ba10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:03.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 --2- 172.21.15.100:0/943559333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0102200 0x7fe7f0111ca0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:03.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 >> 172.21.15.100:0/943559333 conn(0x7fe7f00f7dd0 msgr2=0x7fe7f00f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:03.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 shutdown_connections 2024-09-10T18:25:03.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:03.005+0000 7fe7f7bc8640 1 -- 172.21.15.100:0/943559333 wait complete. 2024-09-10T18:25:03.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:02 smithi100 bash[18597]: cluster 2024-09-10T18:25:01.143022+0000 mgr.y (mgr.14150) 27 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:03.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:25:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:04 smithi100 bash[18597]: audit 2024-09-10T18:25:03.004458+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:04 smithi100 bash[18597]: cephadm 2024-09-10T18:25:03.005252+0000 mgr.y (mgr.14150) 28 : cephadm [INF] Added host smithi151 2024-09-10T18:25:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:04 smithi100 bash[18597]: audit 2024-09-10T18:25:03.006105+0000 mon.a (mon.0) 109 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:04 smithi100 bash[18597]: cluster 2024-09-10T18:25:03.143538+0000 mgr.y (mgr.14150) 29 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:04 smithi100 bash[18597]: audit 2024-09-10T18:25:03.313742+0000 mon.a (mon.0) 110 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:06.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:05 smithi100 bash[18597]: audit 2024-09-10T18:25:04.606342+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:06.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:05 smithi100 bash[18597]: audit 2024-09-10T18:25:05.266141+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:07.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:06 smithi100 bash[18597]: cluster 2024-09-10T18:25:05.143992+0000 mgr.y (mgr.14150) 30 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:08.442 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:25:09.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:08 smithi100 bash[18597]: cluster 2024-09-10T18:25:07.144396+0000 mgr.y (mgr.14150) 31 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:10 smithi100 bash[18597]: cluster 2024-09-10T18:25:09.144802+0000 mgr.y (mgr.14150) 32 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- 172.21.15.100:0/3732980997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 msgr2=0x7f324c0fd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 --2- 172.21.15.100:0/3732980997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c0fd620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f323c004850 tx=0x7f323c009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- 172.21.15.100:0/3732980997 shutdown_connections 2024-09-10T18:25:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 --2- 172.21.15.100:0/3732980997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c0fd620 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- 172.21.15.100:0/3732980997 >> 172.21.15.100:0/3732980997 conn(0x7f324c0f6ce0 msgr2=0x7f324c0f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:11.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- 172.21.15.100:0/3732980997 shutdown_connections 2024-09-10T18:25:11.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- 172.21.15.100:0/3732980997 wait complete. 2024-09-10T18:25:11.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 Processor -- start 2024-09-10T18:25:11.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- start start 2024-09-10T18:25:11.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c111d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:11.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c112250 con 0x7f324c0fd250 2024-09-10T18:25:11.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f324b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c111d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:11.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f324b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c111d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49516/0 (socket says 172.21.15.100:49516) 2024-09-10T18:25:11.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f324b7fe640 1 -- 172.21.15.100:0/3595881540 learned_addr learned my addr 172.21.15.100:0/3595881540 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:25:11.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f324b7fe640 1 -- 172.21.15.100:0/3595881540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c005040 con 0x7f324c0fd250 2024-09-10T18:25:11.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f324b7fe640 1 --2- 172.21.15.100:0/3595881540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c111d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f323c003080 tx=0x7f323c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:11.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3243fff640 1 -- 172.21.15.100:0/3595881540 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f323c038070 con 0x7f324c0fd250 2024-09-10T18:25:11.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3243fff640 1 -- 172.21.15.100:0/3595881540 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f323c0093b0 con 0x7f324c0fd250 2024-09-10T18:25:11.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c112450 con 0x7f324c0fd250 2024-09-10T18:25:11.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c112840 con 0x7f324c0fd250 2024-09-10T18:25:11.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.509+0000 7f3243fff640 1 -- 172.21.15.100:0/3595881540 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f323c03c410 con 0x7f324c0fd250 2024-09-10T18:25:11.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.513+0000 7f3243fff640 1 -- 172.21.15.100:0/3595881540 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f323c030020 con 0x7f324c0fd250 2024-09-10T18:25:11.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.513+0000 7f3243fff640 1 --2- 172.21.15.100:0/3595881540 >> v2:172.21.15.100:6800/3986579393 conn(0x7f322c039560 0x7f322c03ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:11.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.513+0000 7f3243fff640 1 -- 172.21.15.100:0/3595881540 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f323c06bcb0 con 0x7f324c0fd250 2024-09-10T18:25:11.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.513+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c065d60 con 0x7f324c0fd250 2024-09-10T18:25:11.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.513+0000 7f324affd640 1 --2- 172.21.15.100:0/3595881540 >> v2:172.21.15.100:6800/3986579393 conn(0x7f322c039560 0x7f322c03ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:11.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.513+0000 7f324affd640 1 --2- 172.21.15.100:0/3595881540 >> v2:172.21.15.100:6800/3986579393 conn(0x7f322c039560 0x7f322c03ba10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3234006fd0 tx=0x7f3234006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:11.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.517+0000 7f3243fff640 1 -- 172.21.15.100:0/3595881540 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f323c041080 con 0x7f324c0fd250 2024-09-10T18:25:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.701+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f324c10d3b0 con 0x7f322c039560 2024-09-10T18:25:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.701+0000 7f3243fff640 1 -- 172.21.15.100:0/3595881540 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f324c10d3b0 con 0x7f322c039560 2024-09-10T18:25:11.708 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:25:11.708 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 >> v2:172.21.15.100:6800/3986579393 conn(0x7f322c039560 msgr2=0x7f322c03ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 --2- 172.21.15.100:0/3595881540 >> v2:172.21.15.100:6800/3986579393 conn(0x7f322c039560 0x7f322c03ba10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3234006fd0 tx=0x7f3234006db0 comp rx=0 tx=0).stop 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 msgr2=0x7f324c111d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 --2- 172.21.15.100:0/3595881540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c111d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f323c003080 tx=0x7f323c009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 shutdown_connections 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 --2- 172.21.15.100:0/3595881540 >> v2:172.21.15.100:6800/3986579393 conn(0x7f322c039560 0x7f322c03ba10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 --2- 172.21.15.100:0/3595881540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f324c0fd250 0x7f324c111d10 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f323c003080 tx=0x7f323c009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:11.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 >> 172.21.15.100:0/3595881540 conn(0x7f324c0f6ce0 msgr2=0x7f324c1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:11.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 shutdown_connections 2024-09-10T18:25:11.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:11.705+0000 7f3252226640 1 -- 172.21.15.100:0/3595881540 wait complete. 2024-09-10T18:25:12.326 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:25:12.326 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:25:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: cluster 2024-09-10T18:25:11.145293+0000 mgr.y (mgr.14150) 33 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.706915+0000 mgr.y (mgr.14150) 34 : audit [DBG] from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:25:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.770470+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.774479+0000 mon.a (mon.0) 114 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.780293+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.784483+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.785650+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.787471+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:11.788852+0000 mon.a (mon.0) 119 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: cephadm 2024-09-10T18:25:11.790908+0000 mgr.y (mgr.14150) 35 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:12.302639+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:12.308689+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:12 smithi100 bash[18597]: audit 2024-09-10T18:25:12.315896+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:13 smithi100 bash[18597]: cephadm 2024-09-10T18:25:11.917884+0000 mgr.y (mgr.14150) 36 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:13 smithi100 bash[18597]: cephadm 2024-09-10T18:25:12.040408+0000 mgr.y (mgr.14150) 37 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:25:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:13 smithi100 bash[18597]: cephadm 2024-09-10T18:25:12.168560+0000 mgr.y (mgr.14150) 38 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:25:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:14 smithi100 bash[18597]: cluster 2024-09-10T18:25:13.145794+0000 mgr.y (mgr.14150) 39 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:17.041 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:25:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:16 smithi100 bash[18597]: cluster 2024-09-10T18:25:15.146194+0000 mgr.y (mgr.14150) 40 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:18 smithi100 bash[18597]: cluster 2024-09-10T18:25:17.146625+0000 mgr.y (mgr.14150) 41 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:20.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 -- 172.21.15.100:0/3354791241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 msgr2=0x7f853c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:20.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 --2- 172.21.15.100:0/3354791241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c1025d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f852c009bd0 tx=0x7f852c004d10 comp rx=0 tx=0).stop 2024-09-10T18:25:20.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 -- 172.21.15.100:0/3354791241 shutdown_connections 2024-09-10T18:25:20.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 --2- 172.21.15.100:0/3354791241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c1025d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:20.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 -- 172.21.15.100:0/3354791241 >> 172.21.15.100:0/3354791241 conn(0x7f853c0f7dd0 msgr2=0x7f853c0fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:20.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 -- 172.21.15.100:0/3354791241 shutdown_connections 2024-09-10T18:25:20.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 -- 172.21.15.100:0/3354791241 wait complete. 2024-09-10T18:25:20.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 Processor -- start 2024-09-10T18:25:20.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.001+0000 7f8541972640 1 -- start start 2024-09-10T18:25:20.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f8541972640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:20.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f8541972640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c06ff50 con 0x7f853c102200 2024-09-10T18:25:20.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f853affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:20.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f853affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c06fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36708/0 (socket says 172.21.15.100:36708) 2024-09-10T18:25:20.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f853affd640 1 -- 172.21.15.100:0/518019916 learned_addr learned my addr 172.21.15.100:0/518019916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:25:20.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f853affd640 1 -- 172.21.15.100:0/518019916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0098b0 con 0x7f853c102200 2024-09-10T18:25:20.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f853affd640 1 --2- 172.21.15.100:0/518019916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c06fa10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f852c009ba0 tx=0x7f852c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:20.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f85237fe640 1 -- 172.21.15.100:0/518019916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c032640 con 0x7f853c102200 2024-09-10T18:25:20.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f85237fe640 1 -- 172.21.15.100:0/518019916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f852c032c10 con 0x7f853c102200 2024-09-10T18:25:20.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c070150 con 0x7f853c102200 2024-09-10T18:25:20.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f85237fe640 1 -- 172.21.15.100:0/518019916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c03d960 con 0x7f853c102200 2024-09-10T18:25:20.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c0705a0 con 0x7f853c102200 2024-09-10T18:25:20.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f85237fe640 1 -- 172.21.15.100:0/518019916 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f852c02f030 con 0x7f853c102200 2024-09-10T18:25:20.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f85237fe640 1 --2- 172.21.15.100:0/518019916 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8524039120 0x7f852403b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:20.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f85237fe640 1 -- 172.21.15.100:0/518019916 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f852c06ea20 con 0x7f853c102200 2024-09-10T18:25:20.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f853a7fc640 1 --2- 172.21.15.100:0/518019916 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8524039120 0x7f852403b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:20.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f853a7fc640 1 --2- 172.21.15.100:0/518019916 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8524039120 0x7f852403b5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8530006fd0 tx=0x7f8530006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:20.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.005+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c0fd970 con 0x7f853c102200 2024-09-10T18:25:20.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.013+0000 7f85237fe640 1 -- 172.21.15.100:0/518019916 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f852c03c960 con 0x7f853c102200 2024-09-10T18:25:20.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.193+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f853c05eda0 con 0x7f853c102200 2024-09-10T18:25:20.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.785+0000 7f85237fe640 1 -- 172.21.15.100:0/518019916 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f852c03c380 con 0x7f853c102200 2024-09-10T18:25:20.790 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-09-10T18:25:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8524039120 msgr2=0x7f852403b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 --2- 172.21.15.100:0/518019916 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8524039120 0x7f852403b5d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8530006fd0 tx=0x7f8530006db0 comp rx=0 tx=0).stop 2024-09-10T18:25:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 msgr2=0x7f853c06fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:20.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 --2- 172.21.15.100:0/518019916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c06fa10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f852c009ba0 tx=0x7f852c004d10 comp rx=0 tx=0).stop 2024-09-10T18:25:20.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 shutdown_connections 2024-09-10T18:25:20.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 --2- 172.21.15.100:0/518019916 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8524039120 0x7f852403b5d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:20.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 --2- 172.21.15.100:0/518019916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f853c102200 0x7f853c06fa10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:20.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 >> 172.21.15.100:0/518019916 conn(0x7f853c0f7dd0 msgr2=0x7f853c0f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:20.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 shutdown_connections 2024-09-10T18:25:20.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:20.789+0000 7f8541972640 1 -- 172.21.15.100:0/518019916 wait complete. 2024-09-10T18:25:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:20 smithi100 bash[18597]: cluster 2024-09-10T18:25:19.147055+0000 mgr.y (mgr.14150) 42 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:20 smithi100 bash[18597]: audit 2024-09-10T18:25:20.200774+0000 mon.a (mon.0) 123 : audit [INF] from='client.? 172.21.15.100:0/518019916' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:25:21.487 INFO:tasks.cephadm:Adding mon.a on smithi100 2024-09-10T18:25:21.487 INFO:tasks.cephadm:Adding mon.c on smithi100 2024-09-10T18:25:21.487 INFO:tasks.cephadm:Adding mon.b on smithi151 2024-09-10T18:25:21.487 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi151:172.21.15.151=b' 2024-09-10T18:25:22.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:21 smithi100 bash[18597]: audit 2024-09-10T18:25:20.789811+0000 mon.a (mon.0) 124 : audit [INF] from='client.? 172.21.15.100:0/518019916' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:25:22.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:21 smithi100 bash[18597]: cluster 2024-09-10T18:25:20.789938+0000 mon.a (mon.0) 125 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:22.722 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:23.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:22 smithi100 bash[18597]: cluster 2024-09-10T18:25:21.147567+0000 mgr.y (mgr.14150) 43 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:25.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:24 smithi100 bash[18597]: cluster 2024-09-10T18:25:23.147973+0000 mgr.y (mgr.14150) 44 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:25.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 -- 172.21.15.151:0/318764507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 msgr2=0x7f24b406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:25.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 --2- 172.21.15.151:0/318764507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b406fe00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24a4004850 tx=0x7f24a4009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:25.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 -- 172.21.15.151:0/318764507 shutdown_connections 2024-09-10T18:25:25.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 --2- 172.21.15.151:0/318764507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b406fe00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:25.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 -- 172.21.15.151:0/318764507 >> 172.21.15.151:0/318764507 conn(0x7f24b40f7dd0 msgr2=0x7f24b40fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:25.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 -- 172.21.15.151:0/318764507 shutdown_connections 2024-09-10T18:25:25.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 -- 172.21.15.151:0/318764507 wait complete. 2024-09-10T18:25:25.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 Processor -- start 2024-09-10T18:25:25.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 -- start start 2024-09-10T18:25:25.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b4113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:25.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24bbca2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b4114460 con 0x7f24b406fa30 2024-09-10T18:25:25.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24b9a3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b4113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:25.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24b9a3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b4113f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.151:47928/0 (socket says 172.21.15.151:47928) 2024-09-10T18:25:25.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24b9a3e640 1 -- 172.21.15.151:0/3087225543 learned_addr learned my addr 172.21.15.151:0/3087225543 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:25:25.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.546+0000 7f24b9a3e640 1 -- 172.21.15.151:0/3087225543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4005040 con 0x7f24b406fa30 2024-09-10T18:25:25.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24b9a3e640 1 --2- 172.21.15.151:0/3087225543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b4113f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a4003080 tx=0x7f24a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:25.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24b27fc640 1 -- 172.21.15.151:0/3087225543 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24a4038070 con 0x7f24b406fa30 2024-09-10T18:25:25.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b4114660 con 0x7f24b406fa30 2024-09-10T18:25:25.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b4114a50 con 0x7f24b406fa30 2024-09-10T18:25:25.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24b27fc640 1 -- 172.21.15.151:0/3087225543 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f24a40093b0 con 0x7f24b406fa30 2024-09-10T18:25:25.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24b27fc640 1 -- 172.21.15.151:0/3087225543 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24a403c410 con 0x7f24b406fa30 2024-09-10T18:25:25.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a0005010 con 0x7f24b406fa30 2024-09-10T18:25:25.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24b27fc640 1 -- 172.21.15.151:0/3087225543 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f24a4030020 con 0x7f24b406fa30 2024-09-10T18:25:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24b27fc640 1 --2- 172.21.15.151:0/3087225543 >> v2:172.21.15.100:6800/3986579393 conn(0x7f249c039560 0x7f249c03ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.550+0000 7f24b27fc640 1 -- 172.21.15.151:0/3087225543 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f24a406bb20 con 0x7f24b406fa30 2024-09-10T18:25:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.554+0000 7f24b923d640 1 --2- 172.21.15.151:0/3087225543 >> v2:172.21.15.100:6800/3986579393 conn(0x7f249c039560 0x7f249c03ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:25.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.554+0000 7f24b923d640 1 --2- 172.21.15.151:0/3087225543 >> v2:172.21.15.100:6800/3986579393 conn(0x7f249c039560 0x7f249c03ba10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006fd0 tx=0x7f24a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:25.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.570+0000 7f24b27fc640 1 -- 172.21.15.151:0/3087225543 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f24a403aaf0 con 0x7f24b406fa30 2024-09-10T18:25:25.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.754+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi151:172.21.15.151=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f24a0000ca0 con 0x7f249c039560 2024-09-10T18:25:25.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24b27fc640 1 -- 172.21.15.151:0/3087225543 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f24a0000ca0 con 0x7f249c039560 2024-09-10T18:25:25.775 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mon update... 2024-09-10T18:25:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 >> v2:172.21.15.100:6800/3986579393 conn(0x7f249c039560 msgr2=0x7f249c03ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 --2- 172.21.15.151:0/3087225543 >> v2:172.21.15.100:6800/3986579393 conn(0x7f249c039560 0x7f249c03ba10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f24a8006fd0 tx=0x7f24a8006db0 comp rx=0 tx=0).stop 2024-09-10T18:25:25.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 msgr2=0x7f24b4113f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 --2- 172.21.15.151:0/3087225543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b4113f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a4003080 tx=0x7f24a4009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 shutdown_connections 2024-09-10T18:25:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 --2- 172.21.15.151:0/3087225543 >> v2:172.21.15.100:6800/3986579393 conn(0x7f249c039560 0x7f249c03ba10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 --2- 172.21.15.151:0/3087225543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24b406fa30 0x7f24b4113f20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 >> 172.21.15.151:0/3087225543 conn(0x7f24b40f7dd0 msgr2=0x7f24b4104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:25.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 shutdown_connections 2024-09-10T18:25:25.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:25.770+0000 7f24bbca2640 1 -- 172.21.15.151:0/3087225543 wait complete. 2024-09-10T18:25:26.380 DEBUG:teuthology.orchestra.run.smithi100:mon.c> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:25:26.382 DEBUG:teuthology.orchestra.run.smithi151:mon.b> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:25:26.385 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:25:26.385 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:25:26.551 INFO:journalctl@ceph.mon.c.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:25:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: cluster 2024-09-10T18:25:25.148369+0000 mgr.y (mgr.14150) 45 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:25.763095+0000 mgr.y (mgr.14150) 46 : audit [DBG] from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi151:172.21.15.151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:25:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: cephadm 2024-09-10T18:25:25.768240+0000 mgr.y (mgr.14150) 47 : cephadm [INF] Saving service mon spec with placement smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi151:172.21.15.151=b;count:3 2024-09-10T18:25:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:25.773803+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:25.775492+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:26.156052+0000 mon.a (mon.0) 128 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:26.157242+0000 mon.a (mon.0) 129 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:26.164069+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:26.165946+0000 mon.a (mon.0) 131 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:26 smithi100 bash[18597]: audit 2024-09-10T18:25:26.167350+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:27.200 INFO:journalctl@ceph.mon.b.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:25:27.605 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:27 smithi100 bash[18597]: cephadm 2024-09-10T18:25:26.168718+0000 mgr.y (mgr.14150) 48 : cephadm [INF] Deploying daemon mon.b on smithi151 2024-09-10T18:25:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:28 smithi100 bash[18597]: cluster 2024-09-10T18:25:27.148772+0000 mgr.y (mgr.14150) 49 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:30 smithi100 bash[18597]: cluster 2024-09-10T18:25:29.149173+0000 mgr.y (mgr.14150) 50 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:32.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.182+0000 7f9bf75fd640 1 -- 172.21.15.151:0/1021709823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 msgr2=0x7f9bf0104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.182+0000 7f9bf75fd640 1 --2- 172.21.15.151:0/1021709823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf0104d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9bec004850 tx=0x7f9bec009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.182+0000 7f9bf75fd640 1 -- 172.21.15.151:0/1021709823 shutdown_connections 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.182+0000 7f9bf75fd640 1 --2- 172.21.15.151:0/1021709823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf0104d10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.182+0000 7f9bf75fd640 1 -- 172.21.15.151:0/1021709823 >> 172.21.15.151:0/1021709823 conn(0x7f9bf0067750 msgr2=0x7f9bf0067b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.182+0000 7f9bf75fd640 1 -- 172.21.15.151:0/1021709823 shutdown_connections 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.182+0000 7f9bf75fd640 1 -- 172.21.15.151:0/1021709823 wait complete. 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf75fd640 1 Processor -- start 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf75fd640 1 -- start start 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf75fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf01057a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:32.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf75fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0105ce0 con 0x7f9bf0104940 2024-09-10T18:25:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf5399640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf01057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf5399640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf01057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.151:37332/0 (socket says 172.21.15.151:37332) 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf5399640 1 -- 172.21.15.151:0/3096010324 learned_addr learned my addr 172.21.15.151:0/3096010324 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf5399640 1 -- 172.21.15.151:0/3096010324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec005040 con 0x7f9bf0104940 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf5399640 1 --2- 172.21.15.151:0/3096010324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf01057a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9bec004820 tx=0x7f9bec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9be5ffb640 1 -- 172.21.15.151:0/3096010324 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bec038070 con 0x7f9bf0104940 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf0105ee0 con 0x7f9bf0104940 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf0106350 con 0x7f9bf0104940 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9be5ffb640 1 -- 172.21.15.151:0/3096010324 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f9bec009430 con 0x7f9bf0104940 2024-09-10T18:25:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9be5ffb640 1 -- 172.21.15.151:0/3096010324 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bec0335c0 con 0x7f9bf0104940 2024-09-10T18:25:32.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9be5ffb640 1 -- 172.21.15.151:0/3096010324 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f9bec030020 con 0x7f9bf0104940 2024-09-10T18:25:32.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9be5ffb640 1 --2- 172.21.15.151:0/3096010324 >> v2:172.21.15.100:6800/3986579393 conn(0x7f9bdc0394d0 0x7f9bdc03b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:32.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9be5ffb640 1 -- 172.21.15.151:0/3096010324 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f9bec06c3a0 con 0x7f9bf0104940 2024-09-10T18:25:32.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.186+0000 7f9bf4b98640 1 --2- 172.21.15.151:0/3096010324 >> v2:172.21.15.100:6800/3986579393 conn(0x7f9bdc0394d0 0x7f9bdc03b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:32.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.190+0000 7f9bf4b98640 1 --2- 172.21.15.151:0/3096010324 >> v2:172.21.15.100:6800/3986579393 conn(0x7f9bdc0394d0 0x7f9bdc03b980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9be0006fd0 tx=0x7f9be0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:32.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.190+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd4005010 con 0x7f9bf0104940 2024-09-10T18:25:32.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.198+0000 7f9be5ffb640 1 -- 172.21.15.151:0/3096010324 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9bec045b70 con 0x7f9bf0104940 2024-09-10T18:25:32.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.462+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9bd40052c0 con 0x7f9bf0104940 2024-09-10T18:25:32.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9be5ffb640 1 -- 172.21.15.151:0/3096010324 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f9bec045390 con 0x7f9bf0104940 2024-09-10T18:25:32.469 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 1 2024-09-10T18:25:32.470 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-10T18:25:32.470 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"fsid":"505ebbde-6fa1-11ef-bcea-c7b262605968","modified":"2024-09-10T18:23:12.325186Z","created":"2024-09-10T18:23:12.325186Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:25:32.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 >> v2:172.21.15.100:6800/3986579393 conn(0x7f9bdc0394d0 msgr2=0x7f9bdc03b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:32.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 --2- 172.21.15.151:0/3096010324 >> v2:172.21.15.100:6800/3986579393 conn(0x7f9bdc0394d0 0x7f9bdc03b980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9be0006fd0 tx=0x7f9be0006db0 comp rx=0 tx=0).stop 2024-09-10T18:25:32.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 msgr2=0x7f9bf01057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:32.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 --2- 172.21.15.151:0/3096010324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf01057a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9bec004820 tx=0x7f9bec009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:32.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 shutdown_connections 2024-09-10T18:25:32.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 --2- 172.21.15.151:0/3096010324 >> v2:172.21.15.100:6800/3986579393 conn(0x7f9bdc0394d0 0x7f9bdc03b980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:32.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 --2- 172.21.15.151:0/3096010324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf0104940 0x7f9bf01057a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:32.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 >> 172.21.15.151:0/3096010324 conn(0x7f9bf0067750 msgr2=0x7f9bf006e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:32.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 shutdown_connections 2024-09-10T18:25:32.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:32.466+0000 7f9bf75fd640 1 -- 172.21.15.151:0/3096010324 wait complete. 2024-09-10T18:25:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:32 smithi100 bash[18597]: cluster 2024-09-10T18:25:31.149595+0000 mgr.y (mgr.14150) 51 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:32 smithi100 bash[18597]: audit 2024-09-10T18:25:32.468638+0000 mon.a (mon.0) 133 : audit [DBG] from='client.? 172.21.15.151:0/3096010324' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:25:33.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:32 smithi151 systemd[1]: Started Ceph mon.b for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:25:34.208 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:25:34.208 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:25:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:34 smithi100 bash[18597]: audit 2024-09-10T18:25:33.040828+0000 mon.a (mon.0) 134 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:34 smithi100 bash[18597]: audit 2024-09-10T18:25:33.046603+0000 mon.a (mon.0) 135 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:34 smithi100 bash[18597]: audit 2024-09-10T18:25:33.052830+0000 mon.a (mon.0) 136 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:34 smithi100 bash[18597]: audit 2024-09-10T18:25:33.053947+0000 mon.a (mon.0) 137 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:34 smithi100 bash[18597]: audit 2024-09-10T18:25:33.055334+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:34 smithi100 bash[18597]: cephadm 2024-09-10T18:25:33.056856+0000 mgr.y (mgr.14150) 52 : cephadm [INF] Deploying daemon mon.c on smithi100 2024-09-10T18:25:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:34 smithi100 bash[18597]: cluster 2024-09-10T18:25:33.149936+0000 mgr.y (mgr.14150) 53 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.238+0000 7f6975209900 0 set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.238+0000 7f6975209900 0 ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.238+0000 7f6975209900 0 pidfile_write: ignore empty --pid-file 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 0 load: jerasure load: lrc 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: RocksDB version: 6.15.5 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Compile date Sep 9 2024 2024-09-10T18:25:34.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: DB SUMMARY 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: DB Session ID: NMPW87Y37VEIW1O80RVD 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: CURRENT file: CURRENT 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.error_if_exists: 0 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.create_if_missing: 0 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.paranoid_checks: 1 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.env: 0x556dec194440 2024-09-10T18:25:34.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.fs: Posix File System 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.info_log: 0x556dede10ee0 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.statistics: (nil) 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.use_fsync: 0 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_log_file_size: 0 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.allow_fallocate: 1 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:25:34.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.use_direct_reads: 0 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.db_log_dir: 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:25:34.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.write_buffer_manager: 0x556dedece780 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:25:34.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:25:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:25:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.unordered_write: 0 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.row_cache: None 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.wal_filter: None 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.preserve_deletes: 0 2024-09-10T18:25:34.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.two_write_queues: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.atomic_flush: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.log_readahead_size: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:25:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_background_jobs: 2 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_background_compactions: -1 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_subcompactions: 1 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_open_files: -1 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:25:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_background_flushes: -1 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Compression algorithms supported: 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kZSTD supported: 0 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kXpressCompression supported: 0 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kBZip2Compression supported: 0 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kLZ4Compression supported: 1 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kZlibCompression supported: 1 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: kSnappyCompression supported: 1 2024-09-10T18:25:34.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.merge_operator: 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_filter: None 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556deddb9a38) 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cache_index_and_filter_blocks: 1 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:25:34.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: pin_top_level_index_and_filter: 1 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: index_type: 0 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: data_block_index_type: 0 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: index_shortening: 1 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: hash_index_allow_collision: 1 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: checksum: 1 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: no_block_cache: 0 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_cache: 0x556dedded1f0 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_cache_name: BinnedLRUCache 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_cache_options: 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: capacity : 536870912 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: num_shard_bits : 4 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: strict_capacity_limit : 0 2024-09-10T18:25:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: high_pri_pool_ratio: 0.000 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_cache_compressed: (nil) 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: persistent_cache: (nil) 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_size: 4096 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_size_deviation: 10 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_restart_interval: 16 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: index_block_restart_interval: 1 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: metadata_block_size: 4096 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: partition_filters: 0 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: use_delta_encoding: 1 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: whole_key_filtering: 1 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: verify_compression: 0 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: read_amp_bytes_per_bit: 0 2024-09-10T18:25:34.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: format_version: 4 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: enable_index_compression: 1 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: block_align: 0 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression: NoCompression 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.num_levels: 7 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:25:34.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:25:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:25:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:25:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.inplace_update_support: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.bloom_locality: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.max_successive_merges: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:25:34.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.ttl: 2592000 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.enable_blob_files: false 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.min_blob_size: 0 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.242+0000 7f6975209900 4 rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1725992734247604, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:25:34.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 3 rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1725992734247987, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725992734, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f3470023-91cb-42a3-8a1e-7e828dfce302", "db_session_id": "NMPW87Y37VEIW1O80RVD"}} 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1725992734248550, "job": 1, "event": "recovery_finished"} 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x556dede22700 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f6975209900 4 rocksdb: DB pointer 0x556deddf0000 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f696afd5640 4 rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.246+0000 7f696afd5640 4 rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ** DB Stats ** 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:25:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ** Compaction Stats [default] ** 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ** Compaction Stats [default] ** 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:25:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Interval compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ** Compaction Stats [default] ** 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:25:34.438 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ** Compaction Stats [default] ** 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:25:34.439 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.250+0000 7f6975209900 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.250+0000 7f6975209900 0 using public_addr v2:172.21.15.151:0/0 -> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.250+0000 7f6975209900 0 starting mon.b rank -1 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.250+0000 7f6975209900 1 mon.b@-1(???) e0 preinit fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 0 mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 0 mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: e1 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: legacy client fscid: -1 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: 2024-09-10T18:25:34.440 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: No filesystems configured 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.278+0000 7f696dfdb640 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.282+0000 7f696dfdb640 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.282+0000 7f696dfdb640 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.282+0000 7f696dfdb640 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.282+0000 7f696dfdb640 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:41.138773+0000 mgr.y (mgr.14150) 10 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:41.829558+0000 mon.a (mon.0) 98 : audit [INF] from='client.? 172.21.15.100:0/1047551919' entity='client.admin' 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:41.850730+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:34.441 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:41.856549+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:43.139236+0000 mgr.y (mgr.14150) 11 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:45.139703+0000 mgr.y (mgr.14150) 12 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:47.140129+0000 mgr.y (mgr.14150) 13 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:49.140555+0000 mgr.y (mgr.14150) 14 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.408512+0000 mon.a (mon.0) 101 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.409787+0000 mon.a (mon.0) 102 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.412365+0000 mon.a (mon.0) 103 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.413811+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.403788+0000 mgr.y (mgr.14150) 15 : audit [DBG] from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:24:50.415507+0000 mgr.y (mgr.14150) 16 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:24:50.526257+0000 mgr.y (mgr.14150) 17 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:24:50.636936+0000 mgr.y (mgr.14150) 18 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:25:34.442 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:24:50.785980+0000 mgr.y (mgr.14150) 19 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.907027+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.910308+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:50.918972+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:51.141029+0000 mgr.y (mgr.14150) 20 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:53.141485+0000 mgr.y (mgr.14150) 21 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:55.141841+0000 mgr.y (mgr.14150) 22 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:57.142257+0000 mgr.y (mgr.14150) 23 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:24:59.142654+0000 mgr.y (mgr.14150) 24 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:24:59.151001+0000 mgr.y (mgr.14150) 25 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:00.398364+0000 mgr.y (mgr.14150) 26 : cephadm [INF] Deploying cephadm binary to smithi151 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:01.143022+0000 mgr.y (mgr.14150) 27 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:03.004458+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:03.005252+0000 mgr.y (mgr.14150) 28 : cephadm [INF] Added host smithi151 2024-09-10T18:25:34.443 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:03.006105+0000 mon.a (mon.0) 109 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:03.143538+0000 mgr.y (mgr.14150) 29 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:03.313742+0000 mon.a (mon.0) 110 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:04.606342+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:05.266141+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:05.143992+0000 mgr.y (mgr.14150) 30 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:07.144396+0000 mgr.y (mgr.14150) 31 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:09.144802+0000 mgr.y (mgr.14150) 32 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:11.145293+0000 mgr.y (mgr.14150) 33 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.706915+0000 mgr.y (mgr.14150) 34 : audit [DBG] from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.770470+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.774479+0000 mon.a (mon.0) 114 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.444 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.780293+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.784483+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.785650+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.787471+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:11.788852+0000 mon.a (mon.0) 119 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:11.790908+0000 mgr.y (mgr.14150) 35 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:12.302639+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:12.308689+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:12.315896+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:11.917884+0000 mgr.y (mgr.14150) 36 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:12.040408+0000 mgr.y (mgr.14150) 37 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:12.168560+0000 mgr.y (mgr.14150) 38 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:13.145794+0000 mgr.y (mgr.14150) 39 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:15.146194+0000 mgr.y (mgr.14150) 40 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.445 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:17.146625+0000 mgr.y (mgr.14150) 41 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:19.147055+0000 mgr.y (mgr.14150) 42 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:20.200774+0000 mon.a (mon.0) 123 : audit [INF] from='client.? 172.21.15.100:0/518019916' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:20.789811+0000 mon.a (mon.0) 124 : audit [INF] from='client.? 172.21.15.100:0/518019916' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:20.789938+0000 mon.a (mon.0) 125 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:21.147567+0000 mgr.y (mgr.14150) 43 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:23.147973+0000 mgr.y (mgr.14150) 44 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:25.148369+0000 mgr.y (mgr.14150) 45 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:25.763095+0000 mgr.y (mgr.14150) 46 : audit [DBG] from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi151:172.21.15.151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:25.768240+0000 mgr.y (mgr.14150) 47 : cephadm [INF] Saving service mon spec with placement smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi151:172.21.15.151=b;count:3 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:25.773803+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:25.775492+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:26.156052+0000 mon.a (mon.0) 128 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:34.446 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:26.157242+0000 mon.a (mon.0) 129 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:26.164069+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:26.165946+0000 mon.a (mon.0) 131 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:26.167350+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:26.168718+0000 mgr.y (mgr.14150) 48 : cephadm [INF] Deploying daemon mon.b on smithi151 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:27.148772+0000 mgr.y (mgr.14150) 49 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:29.149173+0000 mgr.y (mgr.14150) 50 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:31.149595+0000 mgr.y (mgr.14150) 51 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:32.468638+0000 mon.a (mon.0) 133 : audit [DBG] from='client.? 172.21.15.151:0/3096010324' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:33.040828+0000 mon.a (mon.0) 134 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:33.046603+0000 mon.a (mon.0) 135 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:33.052830+0000 mon.a (mon.0) 136 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:33.053947+0000 mon.a (mon.0) 137 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: audit 2024-09-10T18:25:33.055334+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:34.447 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cephadm 2024-09-10T18:25:33.056856+0000 mgr.y (mgr.14150) 52 : cephadm [INF] Deploying daemon mon.c on smithi100 2024-09-10T18:25:34.448 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: cluster 2024-09-10T18:25:33.149936+0000 mgr.y (mgr.14150) 53 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:34.448 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:34 smithi151 bash[22565]: debug 2024-09-10T18:25:34.290+0000 7f696dfdb640 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:25:37.924 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:25:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:38 smithi100 bash[31607]: debug 2024-09-10T18:25:38.613+0000 7fd27aff8640 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:25:39.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:34.305981+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:25:39.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:34.306366+0000 mon.a (mon.0) 141 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:39.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:34.308507+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:35.150232+0000 mgr.y (mgr.14150) 54 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:39.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:35.300147+0000 mon.a (mon.0) 143 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:36.300237+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:36.300700+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:37.150539+0000 mgr.y (mgr.14150) 55 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:37.300153+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:38.300748+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:38.620550+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:39.300916+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:39.312105+0000 mon.a (mon.0) 149 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:39.323175+0000 mon.a (mon.0) 150 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:39.324258+0000 mon.a (mon.0) 151 : cluster [DBG] fsmap 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:39.324315+0000 mon.a (mon.0) 152 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:39.324578+0000 mon.a (mon.0) 153 : cluster [DBG] mgrmap e13: y(active, since 80s) 2024-09-10T18:25:39.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: cluster 2024-09-10T18:25:39.324849+0000 mon.a (mon.0) 154 : cluster [INF] overall HEALTH_OK 2024-09-10T18:25:39.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:39.331771+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:39.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:39.338664+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:39.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:39.344346+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:39.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:39 smithi151 bash[22565]: audit 2024-09-10T18:25:39.350787+0000 mon.a (mon.0) 158 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:34.305981+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:25:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:34.306366+0000 mon.a (mon.0) 141 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:34.308507+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:35.150232+0000 mgr.y (mgr.14150) 54 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:35.300147+0000 mon.a (mon.0) 143 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:36.300237+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:36.300700+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:37.150539+0000 mgr.y (mgr.14150) 55 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:37.300153+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:38.300748+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:38.620550+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:39.300916+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:39.312105+0000 mon.a (mon.0) 149 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:25:39.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:39.323175+0000 mon.a (mon.0) 150 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:39.324258+0000 mon.a (mon.0) 151 : cluster [DBG] fsmap 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:39.324315+0000 mon.a (mon.0) 152 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:39.324578+0000 mon.a (mon.0) 153 : cluster [DBG] mgrmap e13: y(active, since 80s) 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: cluster 2024-09-10T18:25:39.324849+0000 mon.a (mon.0) 154 : cluster [INF] overall HEALTH_OK 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:39.331771+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:39.338664+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:39.344346+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:39 smithi100 bash[18597]: audit 2024-09-10T18:25:39.350787+0000 mon.a (mon.0) 158 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:40.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:40 smithi151 bash[22565]: cluster 2024-09-10T18:25:39.150934+0000 mgr.y (mgr.14150) 56 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:40.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:40 smithi151 bash[22565]: audit 2024-09-10T18:25:39.395291+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:40.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:40 smithi151 bash[22565]: audit 2024-09-10T18:25:39.620660+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:40.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:40 smithi151 bash[22565]: audit 2024-09-10T18:25:40.300990+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:40 smithi100 bash[18597]: cluster 2024-09-10T18:25:39.150934+0000 mgr.y (mgr.14150) 56 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:40 smithi100 bash[18597]: audit 2024-09-10T18:25:39.395291+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:40 smithi100 bash[18597]: audit 2024-09-10T18:25:39.620660+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:40 smithi100 bash[18597]: audit 2024-09-10T18:25:40.300990+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:40.627170+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:40.629266+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:40.629700+0000 mon.a (mon.0) 164 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:40.650817+0000 mon.a (mon.0) 165 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:40.651208+0000 mon.a (mon.0) 166 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:41.151419+0000 mgr.y (mgr.14150) 57 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:41.621280+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:42.621729+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:43.151880+0000 mgr.y (mgr.14150) 58 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:43.621692+0000 mon.a (mon.0) 169 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:44.621813+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:45.621990+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.650949+0000 mon.a (mon.0) 172 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.656946+0000 mon.a (mon.0) 173 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.657944+0000 mon.a (mon.0) 174 : cluster [DBG] fsmap 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.657991+0000 mon.a (mon.0) 175 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.658189+0000 mon.a (mon.0) 176 : cluster [DBG] mgrmap e13: y(active, since 86s) 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.658414+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.679193+0000 mon.a (mon.0) 178 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.679254+0000 mon.a (mon.0) 179 : cluster [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.679293+0000 mon.a (mon.0) 180 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] is down (out of quorum) 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:45.684231+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:45.689529+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:45.694376+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:45.699935+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:45.705610+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:45 smithi100 bash[18597]: audit 2024-09-10T18:25:45.707202+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:40.627170+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:40.629266+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:40.629700+0000 mon.a (mon.0) 164 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:40.650817+0000 mon.a (mon.0) 165 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:40.651208+0000 mon.a (mon.0) 166 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:41.151419+0000 mgr.y (mgr.14150) 57 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:41.621280+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:42.621729+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:43.151880+0000 mgr.y (mgr.14150) 58 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:43.621692+0000 mon.a (mon.0) 169 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:44.621813+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:45.621990+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.650949+0000 mon.a (mon.0) 172 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.656946+0000 mon.a (mon.0) 173 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.657944+0000 mon.a (mon.0) 174 : cluster [DBG] fsmap 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.657991+0000 mon.a (mon.0) 175 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.658189+0000 mon.a (mon.0) 176 : cluster [DBG] mgrmap e13: y(active, since 86s) 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.658414+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.679193+0000 mon.a (mon.0) 178 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:25:46.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.679254+0000 mon.a (mon.0) 179 : cluster [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:25:46.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.679293+0000 mon.a (mon.0) 180 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] is down (out of quorum) 2024-09-10T18:25:46.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:45.684231+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:45.689529+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:45.694376+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:45.699935+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:45.705610+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:46.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:45 smithi151 bash[22565]: audit 2024-09-10T18:25:45.707202+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:47.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- 172.21.15.151:0/1369951583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc006160 msgr2=0x7f2adc0085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:47.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 --2- 172.21.15.151:0/1369951583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc006160 0x7f2adc0085c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2af00ff600 tx=0x7f2ae0002040 comp rx=0 tx=0).stop 2024-09-10T18:25:47.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- 172.21.15.151:0/1369951583 shutdown_connections 2024-09-10T18:25:47.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 --2- 172.21.15.151:0/1369951583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc006160 0x7f2adc0085c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- 172.21.15.151:0/1369951583 >> 172.21.15.151:0/1369951583 conn(0x7f2af00fa200 msgr2=0x7f2af00fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:47.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- 172.21.15.151:0/1369951583 shutdown_connections 2024-09-10T18:25:47.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- 172.21.15.151:0/1369951583 wait complete. 2024-09-10T18:25:47.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 Processor -- start 2024-09-10T18:25:47.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- start start 2024-09-10T18:25:47.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 0x7f2af0114130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:47.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0114670 0x7f2af010e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:47.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 0x7f2af010ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:47.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0114ce0 con 0x7f2af0114670 2024-09-10T18:25:47.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af011afc0 con 0x7f2af010e630 2024-09-10T18:25:47.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af797f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af011b100 con 0x7f2adc006160 2024-09-10T18:25:47.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af5f1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 0x7f2af010ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:47.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af4f1a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0114670 0x7f2af010e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:47.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af4f1a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0114670 0x7f2af010e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.151:60408/0 (socket says 172.21.15.151:60408) 2024-09-10T18:25:47.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af5f1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 0x7f2af010ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58616/0 (socket says 172.21.15.151:58616) 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af4f1a640 1 -- 172.21.15.151:0/3620305880 learned_addr learned my addr 172.21.15.151:0/3620305880 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.010+0000 7f2af571b640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 0x7f2af0114130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af571b640 1 -- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 msgr2=0x7f2af0114130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af571b640 1 -- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 msgr2=0x7f2af0114130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af571b640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 0x7f2af0114130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af5f1c640 1 -- 172.21.15.151:0/3620305880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 msgr2=0x7f2af010ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af5f1c640 1 -- 172.21.15.151:0/3620305880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 msgr2=0x7f2af010ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af571b640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 0x7f2af0114130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af5f1c640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 0x7f2af010ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-10T18:25:47.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af5f1c640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 0x7f2af010ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:25:47.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af4f1a640 1 -- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 msgr2=0x7f2af0114130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:25:47.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af4f1a640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 0x7f2af0114130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af4f1a640 1 -- 172.21.15.151:0/3620305880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 msgr2=0x7f2af010ead0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:25:47.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af4f1a640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 0x7f2af010ead0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af4f1a640 1 -- 172.21.15.151:0/3620305880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae0005040 con 0x7f2af0114670 2024-09-10T18:25:47.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af4f1a640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0114670 0x7f2af010e0f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b810 tx=0x7f2aec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:47.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2ae5ffb640 1 -- 172.21.15.151:0/3620305880 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00c730 con 0x7f2af0114670 2024-09-10T18:25:47.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2ae5ffb640 1 -- 172.21.15.151:0/3620305880 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f2aec009240 con 0x7f2af0114670 2024-09-10T18:25:47.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af011b360 con 0x7f2af0114670 2024-09-10T18:25:47.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2ae5ffb640 1 -- 172.21.15.151:0/3620305880 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec014b90 con 0x7f2af0114670 2024-09-10T18:25:47.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af011b900 con 0x7f2af0114670 2024-09-10T18:25:47.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.014+0000 7f2ae5ffb640 1 -- 172.21.15.151:0/3620305880 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f2aec0133e0 con 0x7f2af0114670 2024-09-10T18:25:47.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.018+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af00ffc80 con 0x7f2af0114670 2024-09-10T18:25:47.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.018+0000 7f2ae5ffb640 1 --2- 172.21.15.151:0/3620305880 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2ad40399e0 0x7f2ad403be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:47.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.018+0000 7f2ae5ffb640 1 -- 172.21.15.151:0/3620305880 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f2aec04b660 con 0x7f2af0114670 2024-09-10T18:25:47.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.018+0000 7f2af571b640 1 --2- 172.21.15.151:0/3620305880 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2ad40399e0 0x7f2ad403be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:47.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.018+0000 7f2af571b640 1 --2- 172.21.15.151:0/3620305880 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2ad40399e0 0x7f2ad403be90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0037040 tx=0x7f2ae002d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.026+0000 7f2ae5ffb640 1 -- 172.21.15.151:0/3620305880 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f2aec01a830 con 0x7f2af0114670 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: cluster 2024-09-10T18:25:45.152352+0000 mgr.y (mgr.14150) 59 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:45.709013+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: cephadm 2024-09-10T18:25:45.711489+0000 mgr.y (mgr.14150) 60 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: cephadm 2024-09-10T18:25:45.712330+0000 mgr.y (mgr.14150) 61 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: cephadm 2024-09-10T18:25:45.857774+0000 mgr.y (mgr.14150) 62 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:45.987519+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:45.993206+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:45.999783+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:46.004156+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:46.008329+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:46.050175+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:47.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:46.051672+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:47.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:46.052981+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:47.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:46 smithi100 bash[18597]: audit 2024-09-10T18:25:46.622298+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: cluster 2024-09-10T18:25:45.152352+0000 mgr.y (mgr.14150) 59 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:45.709013+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: cephadm 2024-09-10T18:25:45.711489+0000 mgr.y (mgr.14150) 60 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: cephadm 2024-09-10T18:25:45.712330+0000 mgr.y (mgr.14150) 61 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: cephadm 2024-09-10T18:25:45.857774+0000 mgr.y (mgr.14150) 62 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:45.987519+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:45.993206+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:45.999783+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:46.004156+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:46.008329+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:46.050175+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:47.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:46.051672+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:47.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:46.052981+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:47.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:46 smithi151 bash[22565]: audit 2024-09-10T18:25:46.622298+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.266+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2af005ed70 con 0x7f2af0114670 2024-09-10T18:25:47.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.266+0000 7f2ae5ffb640 1 -- 172.21.15.151:0/3620305880 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1273 (secure 0 0 0) 0x7f2aec04afc0 con 0x7f2af0114670 2024-09-10T18:25:47.271 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-10T18:25:47.271 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"fsid":"505ebbde-6fa1-11ef-bcea-c7b262605968","modified":"2024-09-10T18:25:40.622138Z","created":"2024-09-10T18:23:12.325186Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3301","nonce":0},{"type":"v1","addr":"172.21.15.100:6790","nonce":0}]},"addr":"172.21.15.100:6790/0","public_addr":"172.21.15.100:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:25:47.271 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 3 2024-09-10T18:25:47.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2ad40399e0 msgr2=0x7f2ad403be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:47.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 --2- 172.21.15.151:0/3620305880 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2ad40399e0 0x7f2ad403be90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0037040 tx=0x7f2ae002d040 comp rx=0 tx=0).stop 2024-09-10T18:25:47.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0114670 msgr2=0x7f2af010e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:47.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0114670 0x7f2af010e0f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b810 tx=0x7f2aec0091d0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 shutdown_connections 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 --2- 172.21.15.151:0/3620305880 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2ad40399e0 0x7f2ad403be90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2af010e630 0x7f2af010ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0114670 0x7f2af010e0f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 --2- 172.21.15.151:0/3620305880 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2adc006160 0x7f2af0114130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 >> 172.21.15.151:0/3620305880 conn(0x7f2af00fa200 msgr2=0x7f2af0100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 shutdown_connections 2024-09-10T18:25:47.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:25:47.270+0000 7f2af797f640 1 -- 172.21.15.151:0/3620305880 wait complete. 2024-09-10T18:25:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:42.624166+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-09-10T18:25:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.655237+0000 mon.b (mon.1) 3 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.657189+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-09-10T18:25:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.657953+0000 mon.a (mon.0) 199 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.661391+0000 mon.a (mon.0) 200 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:25:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.667396+0000 mon.a (mon.0) 201 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.667975+0000 mon.a (mon.0) 202 : cluster [DBG] fsmap 2024-09-10T18:25:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.668025+0000 mon.a (mon.0) 203 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.668216+0000 mon.a (mon.0) 204 : cluster [DBG] mgrmap e13: y(active, since 88s) 2024-09-10T18:25:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.668397+0000 mon.a (mon.0) 205 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:25:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.668425+0000 mon.a (mon.0) 206 : cluster [INF] Cluster is now healthy 2024-09-10T18:25:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.672980+0000 mon.a (mon.0) 207 : cluster [INF] overall HEALTH_OK 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:34.305981+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:34.306366+0000 mon.a (mon.0) 141 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:34.308507+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:35.150232+0000 mgr.y (mgr.14150) 54 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:35.300147+0000 mon.a (mon.0) 143 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:36.300237+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:36.300700+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:37.150539+0000 mgr.y (mgr.14150) 55 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:37.300153+0000 mon.a (mon.0) 145 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:38.300748+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:38.620550+0000 mon.a (mon.0) 147 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:39.300916+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:39.312105+0000 mon.a (mon.0) 149 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:39.323175+0000 mon.a (mon.0) 150 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:39.324258+0000 mon.a (mon.0) 151 : cluster [DBG] fsmap 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:39.324315+0000 mon.a (mon.0) 152 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:39.324578+0000 mon.a (mon.0) 153 : cluster [DBG] mgrmap e13: y(active, since 80s) 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:39.324849+0000 mon.a (mon.0) 154 : cluster [INF] overall HEALTH_OK 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:39.331771+0000 mon.a (mon.0) 155 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:39.338664+0000 mon.a (mon.0) 156 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:39.344346+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:39.350787+0000 mon.a (mon.0) 158 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:39.150934+0000 mgr.y (mgr.14150) 56 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:39.395291+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:39.620660+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:40.300990+0000 mon.a (mon.0) 161 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:40.627170+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:40.629266+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:40.629700+0000 mon.a (mon.0) 164 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:40.650817+0000 mon.a (mon.0) 165 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:40.651208+0000 mon.a (mon.0) 166 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:41.151419+0000 mgr.y (mgr.14150) 57 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:41.621280+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:42.621729+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:43.151880+0000 mgr.y (mgr.14150) 58 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:43.621692+0000 mon.a (mon.0) 169 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:44.621813+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.621990+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.650949+0000 mon.a (mon.0) 172 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:25:47.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.656946+0000 mon.a (mon.0) 173 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.657944+0000 mon.a (mon.0) 174 : cluster [DBG] fsmap 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.657991+0000 mon.a (mon.0) 175 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.658189+0000 mon.a (mon.0) 176 : cluster [DBG] mgrmap e13: y(active, since 86s) 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.658414+0000 mon.a (mon.0) 177 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.679193+0000 mon.a (mon.0) 178 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.679254+0000 mon.a (mon.0) 179 : cluster [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.679293+0000 mon.a (mon.0) 180 : cluster [WRN] mon.c (rank 2) addr [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] is down (out of quorum) 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.684231+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.689529+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.694376+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.699935+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.705610+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.707202+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:47.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:45.152352+0000 mgr.y (mgr.14150) 59 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.709013+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cephadm 2024-09-10T18:25:45.711489+0000 mgr.y (mgr.14150) 60 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cephadm 2024-09-10T18:25:45.712330+0000 mgr.y (mgr.14150) 61 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cephadm 2024-09-10T18:25:45.857774+0000 mgr.y (mgr.14150) 62 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.987519+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.993206+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:45.999783+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:46.004156+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:46.008329+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:46.050175+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:46.051672+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:46.052981+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: audit 2024-09-10T18:25:46.622298+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:47.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:42.624166+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.655237+0000 mon.b (mon.1) 3 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.657189+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.657953+0000 mon.a (mon.0) 199 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.661391+0000 mon.a (mon.0) 200 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.667396+0000 mon.a (mon.0) 201 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.667975+0000 mon.a (mon.0) 202 : cluster [DBG] fsmap 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.668025+0000 mon.a (mon.0) 203 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.668216+0000 mon.a (mon.0) 204 : cluster [DBG] mgrmap e13: y(active, since 88s) 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.668397+0000 mon.a (mon.0) 205 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.668425+0000 mon.a (mon.0) 206 : cluster [INF] Cluster is now healthy 2024-09-10T18:25:47.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:47 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.672980+0000 mon.a (mon.0) 207 : cluster [INF] overall HEALTH_OK 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:42.624166+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.655237+0000 mon.b (mon.1) 3 : cluster [INF] mon.b calling monitor election 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.657189+0000 mon.c (mon.2) 2 : cluster [INF] mon.c calling monitor election 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.657953+0000 mon.a (mon.0) 199 : cluster [INF] mon.a calling monitor election 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.661391+0000 mon.a (mon.0) 200 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.667396+0000 mon.a (mon.0) 201 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.667975+0000 mon.a (mon.0) 202 : cluster [DBG] fsmap 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.668025+0000 mon.a (mon.0) 203 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.668216+0000 mon.a (mon.0) 204 : cluster [DBG] mgrmap e13: y(active, since 88s) 2024-09-10T18:25:47.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.668397+0000 mon.a (mon.0) 205 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:25:47.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.668425+0000 mon.a (mon.0) 206 : cluster [INF] Cluster is now healthy 2024-09-10T18:25:47.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:47 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.672980+0000 mon.a (mon.0) 207 : cluster [INF] overall HEALTH_OK 2024-09-10T18:25:48.051 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:25:48.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:25:49.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[18597]: cluster 2024-09-10T18:25:47.152821+0000 mgr.y (mgr.14150) 66 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:49.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[18597]: audit 2024-09-10T18:25:48.320943+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:49.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[18597]: audit 2024-09-10T18:25:48.329414+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:49.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[18597]: audit 2024-09-10T18:25:48.330940+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[18597]: audit 2024-09-10T18:25:48.332127+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[18597]: audit 2024-09-10T18:25:48.333186+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[18597]: audit 2024-09-10T18:25:48.622572+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[31607]: cluster 2024-09-10T18:25:47.152821+0000 mgr.y (mgr.14150) 66 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[31607]: audit 2024-09-10T18:25:48.320943+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[31607]: audit 2024-09-10T18:25:48.329414+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[31607]: audit 2024-09-10T18:25:48.330940+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[31607]: audit 2024-09-10T18:25:48.332127+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[31607]: audit 2024-09-10T18:25:48.333186+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:49.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:48 smithi100 bash[31607]: audit 2024-09-10T18:25:48.622572+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:49.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:48 smithi151 bash[22565]: cluster 2024-09-10T18:25:47.152821+0000 mgr.y (mgr.14150) 66 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:49.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:48 smithi151 bash[22565]: audit 2024-09-10T18:25:48.320943+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:49.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:48 smithi151 bash[22565]: audit 2024-09-10T18:25:48.329414+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:49.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:48 smithi151 bash[22565]: audit 2024-09-10T18:25:48.330940+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:49.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:48 smithi151 bash[22565]: audit 2024-09-10T18:25:48.332127+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:49.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:48 smithi151 bash[22565]: audit 2024-09-10T18:25:48.333186+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:49.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:48 smithi151 bash[22565]: audit 2024-09-10T18:25:48.622572+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:25:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:49 smithi100 bash[18597]: cephadm 2024-09-10T18:25:48.330497+0000 mgr.y (mgr.14150) 67 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-09-10T18:25:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:49 smithi100 bash[18597]: cephadm 2024-09-10T18:25:48.334479+0000 mgr.y (mgr.14150) 68 : cephadm [INF] Reconfiguring daemon mon.c on smithi100 2024-09-10T18:25:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:49 smithi100 bash[31607]: cephadm 2024-09-10T18:25:48.330497+0000 mgr.y (mgr.14150) 67 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-09-10T18:25:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:49 smithi100 bash[31607]: cephadm 2024-09-10T18:25:48.334479+0000 mgr.y (mgr.14150) 68 : cephadm [INF] Reconfiguring daemon mon.c on smithi100 2024-09-10T18:25:50.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:49 smithi151 bash[22565]: cephadm 2024-09-10T18:25:48.330497+0000 mgr.y (mgr.14150) 67 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-09-10T18:25:50.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:49 smithi151 bash[22565]: cephadm 2024-09-10T18:25:48.334479+0000 mgr.y (mgr.14150) 68 : cephadm [INF] Reconfiguring daemon mon.c on smithi100 2024-09-10T18:25:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[18597]: cluster 2024-09-10T18:25:49.153329+0000 mgr.y (mgr.14150) 69 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[18597]: audit 2024-09-10T18:25:50.509002+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[18597]: audit 2024-09-10T18:25:50.513514+0000 mon.a (mon.0) 215 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[18597]: audit 2024-09-10T18:25:50.514981+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[18597]: audit 2024-09-10T18:25:50.516209+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[18597]: audit 2024-09-10T18:25:50.517410+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[31607]: cluster 2024-09-10T18:25:49.153329+0000 mgr.y (mgr.14150) 69 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[31607]: audit 2024-09-10T18:25:50.509002+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[31607]: audit 2024-09-10T18:25:50.513514+0000 mon.a (mon.0) 215 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[31607]: audit 2024-09-10T18:25:50.514981+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[31607]: audit 2024-09-10T18:25:50.516209+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:50 smithi100 bash[31607]: audit 2024-09-10T18:25:50.517410+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:51.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:50 smithi151 bash[22565]: cluster 2024-09-10T18:25:49.153329+0000 mgr.y (mgr.14150) 69 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:51.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:50 smithi151 bash[22565]: audit 2024-09-10T18:25:50.509002+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:51.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:50 smithi151 bash[22565]: audit 2024-09-10T18:25:50.513514+0000 mon.a (mon.0) 215 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:51.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:50 smithi151 bash[22565]: audit 2024-09-10T18:25:50.514981+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:25:51.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:50 smithi151 bash[22565]: audit 2024-09-10T18:25:50.516209+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:25:51.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:50 smithi151 bash[22565]: audit 2024-09-10T18:25:50.517410+0000 mon.a (mon.0) 218 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:52.036 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:51 smithi151 bash[22565]: cephadm 2024-09-10T18:25:50.514577+0000 mgr.y (mgr.14150) 70 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-09-10T18:25:52.036 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:51 smithi151 bash[22565]: cephadm 2024-09-10T18:25:50.518824+0000 mgr.y (mgr.14150) 71 : cephadm [INF] Reconfiguring daemon mon.b on smithi151 2024-09-10T18:25:52.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:51 smithi100 bash[18597]: cephadm 2024-09-10T18:25:50.514577+0000 mgr.y (mgr.14150) 70 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-09-10T18:25:52.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:51 smithi100 bash[18597]: cephadm 2024-09-10T18:25:50.518824+0000 mgr.y (mgr.14150) 71 : cephadm [INF] Reconfiguring daemon mon.b on smithi151 2024-09-10T18:25:52.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:51 smithi100 bash[31607]: cephadm 2024-09-10T18:25:50.514577+0000 mgr.y (mgr.14150) 70 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-09-10T18:25:52.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:51 smithi100 bash[31607]: cephadm 2024-09-10T18:25:50.518824+0000 mgr.y (mgr.14150) 71 : cephadm [INF] Reconfiguring daemon mon.b on smithi151 2024-09-10T18:25:52.778 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:25:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[18597]: cluster 2024-09-10T18:25:51.153809+0000 mgr.y (mgr.14150) 72 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[18597]: audit 2024-09-10T18:25:52.514212+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[18597]: audit 2024-09-10T18:25:52.521235+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[18597]: audit 2024-09-10T18:25:52.523358+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[18597]: audit 2024-09-10T18:25:52.526627+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[18597]: audit 2024-09-10T18:25:52.528418+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[18597]: audit 2024-09-10T18:25:52.536548+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[31607]: cluster 2024-09-10T18:25:51.153809+0000 mgr.y (mgr.14150) 72 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[31607]: audit 2024-09-10T18:25:52.514212+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[31607]: audit 2024-09-10T18:25:52.521235+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[31607]: audit 2024-09-10T18:25:52.523358+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[31607]: audit 2024-09-10T18:25:52.526627+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[31607]: audit 2024-09-10T18:25:52.528418+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:53.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:52 smithi100 bash[31607]: audit 2024-09-10T18:25:52.536548+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:52 smithi151 bash[22565]: cluster 2024-09-10T18:25:51.153809+0000 mgr.y (mgr.14150) 72 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:53.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:52 smithi151 bash[22565]: audit 2024-09-10T18:25:52.514212+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:52 smithi151 bash[22565]: audit 2024-09-10T18:25:52.521235+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:53.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:52 smithi151 bash[22565]: audit 2024-09-10T18:25:52.523358+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:25:53.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:52 smithi151 bash[22565]: audit 2024-09-10T18:25:52.526627+0000 mon.a (mon.0) 222 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:53.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:52 smithi151 bash[22565]: audit 2024-09-10T18:25:52.528418+0000 mon.a (mon.0) 223 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:25:53.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:52 smithi151 bash[22565]: audit 2024-09-10T18:25:52.536548+0000 mon.a (mon.0) 224 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:25:55.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:54 smithi100 bash[18597]: cluster 2024-09-10T18:25:53.154286+0000 mgr.y (mgr.14150) 73 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:55.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:54 smithi100 bash[31607]: cluster 2024-09-10T18:25:53.154286+0000 mgr.y (mgr.14150) 73 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:55.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:54 smithi151 bash[22565]: cluster 2024-09-10T18:25:53.154286+0000 mgr.y (mgr.14150) 73 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:55.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- 172.21.15.100:0/482249614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 msgr2=0x7f1fcc108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/482249614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 0x7f1fcc108370 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc004950 tx=0x7f1fbc009da0 comp rx=0 tx=0).stop 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- 172.21.15.100:0/482249614 shutdown_connections 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/482249614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 0x7f1fcc108370 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/482249614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fcc101f30 0x7f1fcc0fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/482249614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101590 0x7f1fcc101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- 172.21.15.100:0/482249614 >> 172.21.15.100:0/482249614 conn(0x7f1fcc0724f0 msgr2=0x7f1fcc0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- 172.21.15.100:0/482249614 shutdown_connections 2024-09-10T18:25:55.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- 172.21.15.100:0/482249614 wait complete. 2024-09-10T18:25:55.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 Processor -- start 2024-09-10T18:25:55.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- start start 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 0x7f1fcc112970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fcc101590 0x7f1fcc112eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101f30 0x7f1fcc10cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc10d110 con 0x7f1fcc101f30 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc10d280 con 0x7f1fcc101590 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd14c2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc10d440 con 0x7f1fcc0fbce0 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd0cc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101f30 0x7f1fcc10cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 0x7f1fcc112970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd0cc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101f30 0x7f1fcc10cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51368/0 (socket says 172.21.15.100:51368) 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fd0cc1640 1 -- 172.21.15.100:0/3000596318 learned_addr learned my addr 172.21.15.100:0/3000596318 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.872+0000 7f1fcb7fe640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fcc101590 0x7f1fcc112eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd0cc1640 1 -- 172.21.15.100:0/3000596318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 msgr2=0x7f1fcc112970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd0cc1640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 0x7f1fcc112970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:55.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd0cc1640 1 -- 172.21.15.100:0/3000596318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fcc101590 msgr2=0x7f1fcc112eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:55.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd0cc1640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fcc101590 0x7f1fcc112eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:55.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd0cc1640 1 -- 172.21.15.100:0/3000596318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc005040 con 0x7f1fcc101f30 2024-09-10T18:25:55.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd0cc1640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101f30 0x7f1fcc10cb40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc02d040 tx=0x7f1fbc035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:55.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fc8ff9640 1 -- 172.21.15.100:0/3000596318 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc02ab70 con 0x7f1fcc101f30 2024-09-10T18:25:55.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fc8ff9640 1 -- 172.21.15.100:0/3000596318 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f1fbc02acd0 con 0x7f1fcc101f30 2024-09-10T18:25:55.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc10d6c0 con 0x7f1fcc101f30 2024-09-10T18:25:55.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fc8ff9640 1 -- 172.21.15.100:0/3000596318 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc033690 con 0x7f1fcc101f30 2024-09-10T18:25:55.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc1199a0 con 0x7f1fcc101f30 2024-09-10T18:25:55.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc0fcd10 con 0x7f1fcc101f30 2024-09-10T18:25:55.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fc8ff9640 1 -- 172.21.15.100:0/3000596318 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f1fbc030020 con 0x7f1fcc101f30 2024-09-10T18:25:55.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fc8ff9640 1 --2- 172.21.15.100:0/3000596318 >> v2:172.21.15.100:6800/3986579393 conn(0x7f1fb8039740 0x7f1fb803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:25:55.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fc8ff9640 1 -- 172.21.15.100:0/3000596318 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f1fbc06c300 con 0x7f1fcc101f30 2024-09-10T18:25:55.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.876+0000 7f1fcbfff640 1 --2- 172.21.15.100:0/3000596318 >> v2:172.21.15.100:6800/3986579393 conn(0x7f1fb8039740 0x7f1fb803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:25:55.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.880+0000 7f1fcbfff640 1 --2- 172.21.15.100:0/3000596318 >> v2:172.21.15.100:6800/3986579393 conn(0x7f1fb8039740 0x7f1fb803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4004fd0 tx=0x7f1fc400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:25:55.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:55.884+0000 7f1fc8ff9640 1 -- 172.21.15.100:0/3000596318 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f1fbc033ac0 con 0x7f1fcc101f30 2024-09-10T18:25:56.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.064+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1fcc04bf60 con 0x7f1fcc101f30 2024-09-10T18:25:56.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.064+0000 7f1fc8ff9640 1 -- 172.21.15.100:0/3000596318 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f1fbc044390 con 0x7f1fcc101f30 2024-09-10T18:25:56.066 INFO:teuthology.orchestra.run.smithi100.stdout:# minimal ceph.conf for 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:25:56.066 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-09-10T18:25:56.066 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:25:56.066 INFO:teuthology.orchestra.run.smithi100.stdout: mon_host = [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] 2024-09-10T18:25:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 >> v2:172.21.15.100:6800/3986579393 conn(0x7f1fb8039740 msgr2=0x7f1fb803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/3000596318 >> v2:172.21.15.100:6800/3986579393 conn(0x7f1fb8039740 0x7f1fb803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4004fd0 tx=0x7f1fc400b040 comp rx=0 tx=0).stop 2024-09-10T18:25:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101f30 msgr2=0x7f1fcc10cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:25:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101f30 0x7f1fcc10cb40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc02d040 tx=0x7f1fbc035040 comp rx=0 tx=0).stop 2024-09-10T18:25:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 shutdown_connections 2024-09-10T18:25:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/3000596318 >> v2:172.21.15.100:6800/3986579393 conn(0x7f1fb8039740 0x7f1fb803bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:56.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fcc101f30 0x7f1fcc10cb40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fcc101590 0x7f1fcc112eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 --2- 172.21.15.100:0/3000596318 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fcc0fbce0 0x7f1fcc112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:25:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 >> 172.21.15.100:0/3000596318 conn(0x7f1fcc0724f0 msgr2=0x7f1fcc100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:25:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 shutdown_connections 2024-09-10T18:25:56.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:25:56.068+0000 7f1fd14c2640 1 -- 172.21.15.100:0/3000596318 wait complete. 2024-09-10T18:25:56.712 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:25:56.712 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:25:56.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:25:56.728 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:25:56.729 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:25:56.737 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:25:56.737 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:25:56.754 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:25:56.755 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:25:56.765 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-09-10T18:25:56.765 INFO:tasks.cephadm:Adding mgr.x on smithi151 2024-09-10T18:25:56.765 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi100=y;smithi151=x' 2024-09-10T18:25:57.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:56 smithi100 bash[18597]: cluster 2024-09-10T18:25:55.154782+0000 mgr.y (mgr.14150) 74 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:57.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:56 smithi100 bash[18597]: audit 2024-09-10T18:25:56.066030+0000 mon.a (mon.0) 225 : audit [DBG] from='client.? 172.21.15.100:0/3000596318' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:57.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:56 smithi100 bash[31607]: cluster 2024-09-10T18:25:55.154782+0000 mgr.y (mgr.14150) 74 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:57.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:56 smithi100 bash[31607]: audit 2024-09-10T18:25:56.066030+0000 mon.a (mon.0) 225 : audit [DBG] from='client.? 172.21.15.100:0/3000596318' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:57.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:56 smithi151 bash[22565]: cluster 2024-09-10T18:25:55.154782+0000 mgr.y (mgr.14150) 74 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:57.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:56 smithi151 bash[22565]: audit 2024-09-10T18:25:56.066030+0000 mon.a (mon.0) 225 : audit [DBG] from='client.? 172.21.15.100:0/3000596318' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:25:59.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:25:58 smithi100 bash[18597]: cluster 2024-09-10T18:25:57.155217+0000 mgr.y (mgr.14150) 75 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:59.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:25:58 smithi100 bash[31607]: cluster 2024-09-10T18:25:57.155217+0000 mgr.y (mgr.14150) 75 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:25:59.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:25:58 smithi151 bash[22565]: cluster 2024-09-10T18:25:57.155217+0000 mgr.y (mgr.14150) 75 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:00.475 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:26:00.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:00 smithi151 bash[22565]: cluster 2024-09-10T18:25:59.155669+0000 mgr.y (mgr.14150) 76 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:00 smithi100 bash[18597]: cluster 2024-09-10T18:25:59.155669+0000 mgr.y (mgr.14150) 76 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:01.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:00 smithi100 bash[31607]: cluster 2024-09-10T18:25:59.155669+0000 mgr.y (mgr.14150) 76 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:02 smithi100 bash[18597]: cluster 2024-09-10T18:26:01.156101+0000 mgr.y (mgr.14150) 77 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:03.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:02 smithi100 bash[31607]: cluster 2024-09-10T18:26:01.156101+0000 mgr.y (mgr.14150) 77 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:03.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:02 smithi151 bash[22565]: cluster 2024-09-10T18:26:01.156101+0000 mgr.y (mgr.14150) 77 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:03.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- 172.21.15.151:0/3362357487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 msgr2=0x7f4e800fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 --2- 172.21.15.151:0/3362357487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e800fce40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4e70004950 tx=0x7f4e70009da0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- 172.21.15.151:0/3362357487 shutdown_connections 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 --2- 172.21.15.151:0/3362357487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800fd380 0x7f4e80103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 --2- 172.21.15.151:0/3362357487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e800fce40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 --2- 172.21.15.151:0/3362357487 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e800fb700 0x7f4e800fbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- 172.21.15.151:0/3362357487 >> 172.21.15.151:0/3362357487 conn(0x7f4e800f6ea0 msgr2=0x7f4e800f92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- 172.21.15.151:0/3362357487 shutdown_connections 2024-09-10T18:26:03.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- 172.21.15.151:0/3362357487 wait complete. 2024-09-10T18:26:03.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 Processor -- start 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- start start 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e800fb700 0x7f4e8010bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e8010c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800fd380 0x7f4e801131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80113790 con 0x7f4e800fd380 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80113900 con 0x7f4e800fc9d0 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e884ff640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80113ac0 con 0x7f4e800fb700 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e8010c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e8010c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35834/0 (socket says 172.21.15.151:35834) 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 -- 172.21.15.151:0/1743204868 learned_addr learned my addr 172.21.15.151:0/1743204868 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:26:03.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e8629b640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e800fb700 0x7f4e8010bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:03.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e86a9c640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800fd380 0x7f4e801131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:03.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 -- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e800fb700 msgr2=0x7f4e8010bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:03.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e800fb700 0x7f4e8010bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 -- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800fd380 msgr2=0x7f4e801131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:03.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800fd380 0x7f4e801131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.550+0000 7f4e85a9a640 1 -- 172.21.15.151:0/1743204868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70005040 con 0x7f4e800fc9d0 2024-09-10T18:26:03.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e85a9a640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e8010c1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e700049b0 tx=0x7f4e70009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:26:03.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e76ffd640 1 -- 172.21.15.151:0/1743204868 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7002ab70 con 0x7f4e800fc9d0 2024-09-10T18:26:03.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e76ffd640 1 -- 172.21.15.151:0/1743204868 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f4e7002acd0 con 0x7f4e800fc9d0 2024-09-10T18:26:03.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e800652c0 con 0x7f4e800fc9d0 2024-09-10T18:26:03.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e76ffd640 1 -- 172.21.15.151:0/1743204868 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70032ec0 con 0x7f4e800fc9d0 2024-09-10T18:26:03.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e80065860 con 0x7f4e800fc9d0 2024-09-10T18:26:03.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e76ffd640 1 -- 172.21.15.151:0/1743204868 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) v1 ==== 45514+0+0 (secure 0 0 0) 0x7f4e70030020 con 0x7f4e800fc9d0 2024-09-10T18:26:03.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8004bf60 con 0x7f4e800fc9d0 2024-09-10T18:26:03.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e76ffd640 1 --2- 172.21.15.151:0/1743204868 >> v2:172.21.15.100:6800/3986579393 conn(0x7f4e6c039740 0x7f4e6c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:03.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e76ffd640 1 -- 172.21.15.151:0/1743204868 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f4e7006bba0 con 0x7f4e800fc9d0 2024-09-10T18:26:03.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.554+0000 7f4e8629b640 1 --2- 172.21.15.151:0/1743204868 >> v2:172.21.15.100:6800/3986579393 conn(0x7f4e6c039740 0x7f4e6c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:03.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.558+0000 7f4e8629b640 1 --2- 172.21.15.151:0/1743204868 >> v2:172.21.15.100:6800/3986579393 conn(0x7f4e6c039740 0x7f4e6c03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e8010d120 tx=0x7f4e7c011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:26:03.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.566+0000 7f4e76ffd640 1 -- 172.21.15.151:0/1743204868 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4e700325e0 con 0x7f4e800fc9d0 2024-09-10T18:26:03.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.758+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi151=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e80065a80 con 0x7f4e6c039740 2024-09-10T18:26:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.778+0000 7f4e76ffd640 1 -- 172.21.15.151:0/1743204868 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4e80065a80 con 0x7f4e6c039740 2024-09-10T18:26:03.782 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mgr update... 2024-09-10T18:26:03.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 >> v2:172.21.15.100:6800/3986579393 conn(0x7f4e6c039740 msgr2=0x7f4e6c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:03.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 --2- 172.21.15.151:0/1743204868 >> v2:172.21.15.100:6800/3986579393 conn(0x7f4e6c039740 0x7f4e6c03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e8010d120 tx=0x7f4e7c011040 comp rx=0 tx=0).stop 2024-09-10T18:26:03.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 msgr2=0x7f4e8010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:03.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e8010c1a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e700049b0 tx=0x7f4e70009da0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 shutdown_connections 2024-09-10T18:26:03.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 --2- 172.21.15.151:0/1743204868 >> v2:172.21.15.100:6800/3986579393 conn(0x7f4e6c039740 0x7f4e6c03bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800fd380 0x7f4e801131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e800fc9d0 0x7f4e8010c1a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 --2- 172.21.15.151:0/1743204868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e800fb700 0x7f4e8010bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:03.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 >> 172.21.15.151:0/1743204868 conn(0x7f4e800f6ea0 msgr2=0x7f4e800f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:26:03.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 shutdown_connections 2024-09-10T18:26:03.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:26:03.782+0000 7f4e884ff640 1 -- 172.21.15.151:0/1743204868 wait complete. 2024-09-10T18:26:04.647 DEBUG:teuthology.orchestra.run.smithi151:mgr.x> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:26:04.649 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:26:04.649 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:26:04.649 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:26:04.657 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:26:04.657 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-09-10T18:26:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:26:04.707 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:04.708 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-10T18:26:04.708 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:04.708 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-10 18:16:41.593925891 +0000 2024-09-10T18:26:04.708 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-10 18:16:41.589925983 +0000 2024-09-10T18:26:04.708 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-10 18:16:41.589925983 +0000 2024-09-10T18:26:04.708 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-10T18:26:04.708 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:26:04.720 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-10T18:26:04.720 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-10T18:26:04.720 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000435974 s, 1.2 MB/s 2024-09-10T18:26:04.721 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:26:04.730 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 367 Links: 1 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-10 18:16:42.213911627 +0000 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-10 18:16:42.009916320 +0000 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-10 18:16:42.009916320 +0000 2024-09-10T18:26:04.737 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-10T18:26:04.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:26:04.750 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-10T18:26:04.750 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-10T18:26:04.750 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000341027 s, 1.5 MB/s 2024-09-10T18:26:04.751 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:26:04.757 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-10 18:16:42.661901320 +0000 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-10 18:16:42.457906013 +0000 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-10 18:16:42.457906013 +0000 2024-09-10T18:26:04.763 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-10T18:26:04.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:26:04.775 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-10T18:26:04.775 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-10T18:26:04.775 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000367333 s, 1.4 MB/s 2024-09-10T18:26:04.776 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:26:04.829 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-10 18:16:43.069891934 +0000 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-10 18:16:42.857896812 +0000 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-10 18:16:42.857896812 +0000 2024-09-10T18:26:04.837 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: - 2024-09-10T18:26:04.837 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:26:04.852 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-10T18:26:04.852 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-10T18:26:04.852 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000321896 s, 1.6 MB/s 2024-09-10T18:26:04.853 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:26:04.860 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:26:04.860 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:26:04.867 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:26:04.867 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-09-10T18:26:04.874 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:26:04.874 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:04.874 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-10T18:26:04.874 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:04.875 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-10 18:16:40.543393473 +0000 2024-09-10T18:26:04.875 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-10 18:16:40.539393564 +0000 2024-09-10T18:26:04.875 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-10 18:16:40.539393564 +0000 2024-09-10T18:26:04.875 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-09-10T18:26:04.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:26:04.884 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-10T18:26:04.884 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-10T18:26:04.884 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000429066 s, 1.2 MB/s 2024-09-10T18:26:04.885 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:26:04.893 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-09-10T18:26:04.899 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:26:04.899 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:04.899 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 362 Links: 1 2024-09-10T18:26:04.899 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:04.900 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-10 18:16:41.175378917 +0000 2024-09-10T18:26:04.900 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-10 18:16:40.947384169 +0000 2024-09-10T18:26:04.900 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-10 18:16:40.947384169 +0000 2024-09-10T18:26:04.900 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-09-10T18:26:04.900 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:26:04.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: cluster 2024-09-10T18:26:03.156553+0000 mgr.y (mgr.14150) 78 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:04.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.763345+0000 mgr.y (mgr.14150) 79 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi151=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:26:04.909 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: cephadm 2024-09-10T18:26:03.766720+0000 mgr.y (mgr.14150) 80 : cephadm [INF] Saving service mgr spec with placement smithi100=y;smithi151=x;count:2 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.773231+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.783319+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.786340+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.787976+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.796992+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.799358+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.804689+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.806122+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: audit 2024-09-10T18:26:03.807680+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:04.910 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:04 smithi151 bash[22565]: cephadm 2024-09-10T18:26:03.809326+0000 mgr.y (mgr.14150) 81 : cephadm [INF] Deploying daemon mgr.x on smithi151 2024-09-10T18:26:04.911 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-10T18:26:04.911 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-10T18:26:04.912 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000400575 s, 1.3 MB/s 2024-09-10T18:26:04.912 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:26:04.965 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-09-10T18:26:04.972 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:26:04.973 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:04.973 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-09-10T18:26:04.973 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:04.973 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-10 18:16:41.403373666 +0000 2024-09-10T18:26:04.973 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-10 18:16:41.399373759 +0000 2024-09-10T18:26:04.973 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-10 18:16:41.399373759 +0000 2024-09-10T18:26:04.973 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-09-10T18:26:04.973 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:26:04.983 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-10T18:26:04.984 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-10T18:26:04.984 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000403503 s, 1.3 MB/s 2024-09-10T18:26:04.985 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:26:04.992 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-09-10T18:26:05.043 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:26:05.043 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:26:05.043 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-10T18:26:05.043 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:26:05.044 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-10 18:16:42.039359019 +0000 2024-09-10T18:26:05.044 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-10 18:16:41.827363901 +0000 2024-09-10T18:26:05.044 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-10 18:16:41.827363901 +0000 2024-09-10T18:26:05.044 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-09-10T18:26:05.044 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: cluster 2024-09-10T18:26:03.156553+0000 mgr.y (mgr.14150) 78 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.763345+0000 mgr.y (mgr.14150) 79 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi151=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: cephadm 2024-09-10T18:26:03.766720+0000 mgr.y (mgr.14150) 80 : cephadm [INF] Saving service mgr spec with placement smithi100=y;smithi151=x;count:2 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.773231+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.783319+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.786340+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.787976+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.796992+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.799358+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.804689+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:26:05.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.806122+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: audit 2024-09-10T18:26:03.807680+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[18597]: cephadm 2024-09-10T18:26:03.809326+0000 mgr.y (mgr.14150) 81 : cephadm [INF] Deploying daemon mgr.x on smithi151 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: cluster 2024-09-10T18:26:03.156553+0000 mgr.y (mgr.14150) 78 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.763345+0000 mgr.y (mgr.14150) 79 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi151=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: cephadm 2024-09-10T18:26:03.766720+0000 mgr.y (mgr.14150) 80 : cephadm [INF] Saving service mgr spec with placement smithi100=y;smithi151=x;count:2 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.773231+0000 mon.a (mon.0) 226 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.783319+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.786340+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.787976+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.796992+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.799358+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.804689+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:26:05.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.806122+0000 mon.a (mon.0) 233 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:26:05.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: audit 2024-09-10T18:26:03.807680+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:05.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:04 smithi100 bash[31607]: cephadm 2024-09-10T18:26:03.809326+0000 mgr.y (mgr.14150) 81 : cephadm [INF] Deploying daemon mgr.x on smithi151 2024-09-10T18:26:05.057 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-10T18:26:05.057 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-10T18:26:05.057 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000364453 s, 1.4 MB/s 2024-09-10T18:26:05.058 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:26:05.109 INFO:tasks.cephadm:Deploying osd.0 on smithi100 with /dev/vg_nvme/lv_4... 2024-09-10T18:26:05.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:26:05.463 INFO:journalctl@ceph.mgr.x.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:26:07.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[18597]: cluster 2024-09-10T18:26:05.156917+0000 mgr.y (mgr.14150) 82 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:07.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[18597]: audit 2024-09-10T18:26:06.743898+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[18597]: audit 2024-09-10T18:26:06.753136+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[18597]: audit 2024-09-10T18:26:06.762203+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[18597]: audit 2024-09-10T18:26:06.770101+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[31607]: cluster 2024-09-10T18:26:05.156917+0000 mgr.y (mgr.14150) 82 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:07.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[31607]: audit 2024-09-10T18:26:06.743898+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[31607]: audit 2024-09-10T18:26:06.753136+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[31607]: audit 2024-09-10T18:26:06.762203+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:06 smithi100 bash[31607]: audit 2024-09-10T18:26:06.770101+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:06 smithi151 bash[22565]: cluster 2024-09-10T18:26:05.156917+0000 mgr.y (mgr.14150) 82 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:07.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:06 smithi151 bash[22565]: audit 2024-09-10T18:26:06.743898+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:06 smithi151 bash[22565]: audit 2024-09-10T18:26:06.753136+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:06 smithi151 bash[22565]: audit 2024-09-10T18:26:06.762203+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:07.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:06 smithi151 bash[22565]: audit 2024-09-10T18:26:06.770101+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:08.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:07 smithi100 bash[18597]: audit 2024-09-10T18:26:06.817085+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:08.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:07 smithi100 bash[31607]: audit 2024-09-10T18:26:06.817085+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:08.062 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:07 smithi151 bash[22565]: audit 2024-09-10T18:26:06.817085+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:08.062 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:07 smithi151 bash[23560]: debug 2024-09-10T18:26:07.930+0000 7fc1e01ce640 1 -- 172.21.15.151:0/1420259951 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556524fdc340 con 0x556524fe8c00 2024-09-10T18:26:08.415 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:08 smithi151 bash[23560]: debug 2024-09-10T18:26:08.058+0000 7fc1e2a36ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:26:09.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:08 smithi100 bash[18597]: cluster 2024-09-10T18:26:07.157321+0000 mgr.y (mgr.14150) 83 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:09.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:08 smithi100 bash[31607]: cluster 2024-09-10T18:26:07.157321+0000 mgr.y (mgr.14150) 83 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:09.088 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:08 smithi151 bash[22565]: cluster 2024-09-10T18:26:07.157321+0000 mgr.y (mgr.14150) 83 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:09.088 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:08 smithi151 bash[23560]: debug 2024-09-10T18:26:08.802+0000 7fc1e2a36ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:26:09.089 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:08 smithi151 bash[23560]: debug 2024-09-10T18:26:08.870+0000 7fc1e2a36ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:26:09.089 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:08 smithi151 bash[23560]: debug 2024-09-10T18:26:08.934+0000 7fc1e2a36ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:26:09.392 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:09 smithi151 bash[23560]: debug 2024-09-10T18:26:09.082+0000 7fc1e2a36ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:26:09.392 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:09 smithi151 bash[23560]: debug 2024-09-10T18:26:09.326+0000 7fc1e2a36ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:26:09.665 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:09 smithi151 bash[23560]: debug 2024-09-10T18:26:09.386+0000 7fc1e2a36ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:26:09.665 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:09 smithi151 bash[23560]: debug 2024-09-10T18:26:09.518+0000 7fc1e2a36ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:26:09.840 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:26:10.416 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:10 smithi151 bash[23560]: debug 2024-09-10T18:26:10.146+0000 7fc1e2a36ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:26:10.416 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:10 smithi151 bash[23560]: debug 2024-09-10T18:26:10.218+0000 7fc1e2a36ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:26:10.416 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:10 smithi151 bash[23560]: debug 2024-09-10T18:26:10.290+0000 7fc1e2a36ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:26:10.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:10 smithi151 bash[22565]: cluster 2024-09-10T18:26:09.157767+0000 mgr.y (mgr.14150) 84 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:10.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:10 smithi151 bash[22565]: audit 2024-09-10T18:26:10.698072+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:10.916 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:10 smithi151 bash[23560]: debug 2024-09-10T18:26:10.566+0000 7fc1e2a36ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:26:10.916 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:10 smithi151 bash[23560]: debug 2024-09-10T18:26:10.698+0000 7fc1e2a36ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:26:11.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:10 smithi100 bash[18597]: cluster 2024-09-10T18:26:09.157767+0000 mgr.y (mgr.14150) 84 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:11.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:10 smithi100 bash[18597]: audit 2024-09-10T18:26:10.698072+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:11.051 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:10 smithi100 bash[31607]: cluster 2024-09-10T18:26:09.157767+0000 mgr.y (mgr.14150) 84 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:11.051 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:10 smithi100 bash[31607]: audit 2024-09-10T18:26:10.698072+0000 mon.a (mon.0) 240 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:11.415 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:11 smithi151 bash[23560]: debug 2024-09-10T18:26:11.010+0000 7fc1e2a36ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:26:11.416 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:11 smithi151 bash[23560]: debug 2024-09-10T18:26:11.222+0000 7fc1e2a36ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:26:11.915 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:11 smithi151 bash[23560]: debug 2024-09-10T18:26:11.638+0000 7fc1e2a36ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:26:11.916 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:11 smithi151 bash[23560]: debug 2024-09-10T18:26:11.702+0000 7fc1e2a36ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:26:11.916 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:11 smithi151 bash[23560]: debug 2024-09-10T18:26:11.774+0000 7fc1e2a36ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:26:11.916 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:11 smithi151 bash[23560]: debug 2024-09-10T18:26:11.878+0000 7fc1e2a36ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:26:12.252 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:11 smithi151 bash[23560]: debug 2024-09-10T18:26:11.942+0000 7fc1e2a36ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:26:12.252 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[23560]: debug 2024-09-10T18:26:12.022+0000 7fc1e2a36ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:26:12.252 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[23560]: debug 2024-09-10T18:26:12.170+0000 7fc1e2a36ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:26:12.665 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[23560]: debug 2024-09-10T18:26:12.246+0000 7fc1e2a36ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:26:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: cluster 2024-09-10T18:26:11.158208+0000 mgr.y (mgr.14150) 85 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:11.854525+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:11.862158+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:12.231208+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:12.233005+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:12.244184+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:12.290311+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:12.292002+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[18597]: audit 2024-09-10T18:26:12.293412+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: cluster 2024-09-10T18:26:11.158208+0000 mgr.y (mgr.14150) 85 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:11.854525+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:11.862158+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:12.231208+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:13.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:12.233005+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:13.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:12.244184+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:12.290311+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:26:13.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:12.292002+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:26:13.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:12 smithi100 bash[31607]: audit 2024-09-10T18:26:12.293412+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:13.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: cluster 2024-09-10T18:26:11.158208+0000 mgr.y (mgr.14150) 85 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:11.854525+0000 mon.a (mon.0) 241 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:11.862158+0000 mon.a (mon.0) 242 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:12.231208+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:12.233005+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:12.244184+0000 mon.a (mon.0) 245 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:12.290311+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:12.292002+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:26:13.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:12 smithi151 bash[22565]: audit 2024-09-10T18:26:12.293412+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:13.665 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[23560]: debug 2024-09-10T18:26:13.170+0000 7fc1e2a36ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:26:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:26:13.986 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:26:14.006 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-10T18:26:14.006 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-10T18:26:14.007 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0161598 s, 649 MB/s 2024-09-10T18:26:14.007 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 2024-09-10T18:26:14.007 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-10T18:26:14.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[22565]: cephadm 2024-09-10T18:26:12.289620+0000 mgr.y (mgr.14150) 86 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:26:14.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[22565]: cephadm 2024-09-10T18:26:12.294833+0000 mgr.y (mgr.14150) 87 : cephadm [INF] Reconfiguring daemon mgr.y on smithi100 2024-09-10T18:26:14.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[22565]: audit 2024-09-10T18:26:13.196372+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:26:14.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[22565]: cluster 2024-09-10T18:26:13.196722+0000 mon.a (mon.0) 249 : cluster [DBG] Standby manager daemon x started 2024-09-10T18:26:14.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[22565]: audit 2024-09-10T18:26:13.197119+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:26:14.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[22565]: audit 2024-09-10T18:26:13.198797+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:26:14.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:13 smithi151 bash[22565]: audit 2024-09-10T18:26:13.199509+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:26:14.246 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[31607]: cephadm 2024-09-10T18:26:12.289620+0000 mgr.y (mgr.14150) 86 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:26:14.246 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[31607]: cephadm 2024-09-10T18:26:12.294833+0000 mgr.y (mgr.14150) 87 : cephadm [INF] Reconfiguring daemon mgr.y on smithi100 2024-09-10T18:26:14.246 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[31607]: audit 2024-09-10T18:26:13.196372+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:26:14.246 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[31607]: cluster 2024-09-10T18:26:13.196722+0000 mon.a (mon.0) 249 : cluster [DBG] Standby manager daemon x started 2024-09-10T18:26:14.246 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[31607]: audit 2024-09-10T18:26:13.197119+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:26:14.246 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[31607]: audit 2024-09-10T18:26:13.198797+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:26:14.246 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[31607]: audit 2024-09-10T18:26:13.199509+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:26:14.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[18597]: cephadm 2024-09-10T18:26:12.289620+0000 mgr.y (mgr.14150) 86 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:26:14.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[18597]: cephadm 2024-09-10T18:26:12.294833+0000 mgr.y (mgr.14150) 87 : cephadm [INF] Reconfiguring daemon mgr.y on smithi100 2024-09-10T18:26:14.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[18597]: audit 2024-09-10T18:26:13.196372+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:26:14.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[18597]: cluster 2024-09-10T18:26:13.196722+0000 mon.a (mon.0) 249 : cluster [DBG] Standby manager daemon x started 2024-09-10T18:26:14.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[18597]: audit 2024-09-10T18:26:13.197119+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:26:14.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[18597]: audit 2024-09-10T18:26:13.198797+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:26:14.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:13 smithi100 bash[18597]: audit 2024-09-10T18:26:13.199509+0000 mon.b (mon.1) 7 : audit [DBG] from='mgr.? 172.21.15.151:0/1939416355' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:26:14.966 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-09-10T18:26:15.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:14 smithi151 bash[22565]: cluster 2024-09-10T18:26:13.158673+0000 mgr.y (mgr.14150) 88 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:15.165 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:14 smithi151 bash[22565]: cluster 2024-09-10T18:26:13.878701+0000 mon.a (mon.0) 250 : cluster [DBG] mgrmap e14: y(active, since 114s), standbys: x 2024-09-10T18:26:15.166 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:14 smithi151 bash[22565]: audit 2024-09-10T18:26:13.879059+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:26:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:14 smithi100 bash[18597]: cluster 2024-09-10T18:26:13.158673+0000 mgr.y (mgr.14150) 88 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:14 smithi100 bash[18597]: cluster 2024-09-10T18:26:13.878701+0000 mon.a (mon.0) 250 : cluster [DBG] mgrmap e14: y(active, since 114s), standbys: x 2024-09-10T18:26:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:14 smithi100 bash[18597]: audit 2024-09-10T18:26:13.879059+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:26:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:14 smithi100 bash[31607]: cluster 2024-09-10T18:26:13.158673+0000 mgr.y (mgr.14150) 88 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:14 smithi100 bash[31607]: cluster 2024-09-10T18:26:13.878701+0000 mon.a (mon.0) 250 : cluster [DBG] mgrmap e14: y(active, since 114s), standbys: x 2024-09-10T18:26:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:14 smithi100 bash[31607]: audit 2024-09-10T18:26:13.879059+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:26:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[18597]: audit 2024-09-10T18:26:14.977146+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[18597]: audit 2024-09-10T18:26:14.983002+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[18597]: audit 2024-09-10T18:26:14.985322+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[18597]: audit 2024-09-10T18:26:14.988109+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[18597]: audit 2024-09-10T18:26:14.989445+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[18597]: audit 2024-09-10T18:26:14.997121+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[18597]: cluster 2024-09-10T18:26:15.158952+0000 mgr.y (mgr.14150) 89 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[31607]: audit 2024-09-10T18:26:14.977146+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[31607]: audit 2024-09-10T18:26:14.983002+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[31607]: audit 2024-09-10T18:26:14.985322+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[31607]: audit 2024-09-10T18:26:14.988109+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[31607]: audit 2024-09-10T18:26:14.989445+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[31607]: audit 2024-09-10T18:26:14.997121+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:15 smithi100 bash[31607]: cluster 2024-09-10T18:26:15.158952+0000 mgr.y (mgr.14150) 89 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:16.415 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:15 smithi151 bash[22565]: audit 2024-09-10T18:26:14.977146+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:15 smithi151 bash[22565]: audit 2024-09-10T18:26:14.983002+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:15 smithi151 bash[22565]: audit 2024-09-10T18:26:14.985322+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:16.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:15 smithi151 bash[22565]: audit 2024-09-10T18:26:14.988109+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:16.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:15 smithi151 bash[22565]: audit 2024-09-10T18:26:14.989445+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:16.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:15 smithi151 bash[22565]: audit 2024-09-10T18:26:14.997121+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:16.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:15 smithi151 bash[22565]: cluster 2024-09-10T18:26:15.158952+0000 mgr.y (mgr.14150) 89 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:18.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:18 smithi100 bash[18597]: cluster 2024-09-10T18:26:17.159342+0000 mgr.y (mgr.14150) 90 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:18.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:18 smithi100 bash[31607]: cluster 2024-09-10T18:26:17.159342+0000 mgr.y (mgr.14150) 90 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:18.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:18 smithi151 bash[22565]: cluster 2024-09-10T18:26:17.159342+0000 mgr.y (mgr.14150) 90 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:26:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:20 smithi100 bash[18597]: cluster 2024-09-10T18:26:19.159763+0000 mgr.y (mgr.14150) 91 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:20 smithi100 bash[31607]: cluster 2024-09-10T18:26:19.159763+0000 mgr.y (mgr.14150) 91 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:20.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:20 smithi151 bash[22565]: cluster 2024-09-10T18:26:19.159763+0000 mgr.y (mgr.14150) 91 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:22.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:22 smithi100 bash[18597]: cluster 2024-09-10T18:26:21.160203+0000 mgr.y (mgr.14150) 92 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:22 smithi100 bash[31607]: cluster 2024-09-10T18:26:21.160203+0000 mgr.y (mgr.14150) 92 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:22.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:22 smithi151 bash[22565]: cluster 2024-09-10T18:26:21.160203+0000 mgr.y (mgr.14150) 92 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:22.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 -- 172.21.15.100:0/592272129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7284066c90 msgr2=0x7f72840fd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:22.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 --2- 172.21.15.100:0/592272129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7284066c90 0x7f72840fd1d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f726c004950 tx=0x7f726c009da0 comp rx=0 tx=0).stop 2024-09-10T18:26:22.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 -- 172.21.15.100:0/592272129 shutdown_connections 2024-09-10T18:26:22.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 --2- 172.21.15.100:0/592272129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72841002c0 0x7f7284100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:22.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 --2- 172.21.15.100:0/592272129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72840fd710 0x7f72840ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:22.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 --2- 172.21.15.100:0/592272129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7284066c90 0x7f72840fd1d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:22.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 -- 172.21.15.100:0/592272129 >> 172.21.15.100:0/592272129 conn(0x7f72840f6e90 msgr2=0x7f72840f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:26:22.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 -- 172.21.15.100:0/592272129 shutdown_connections 2024-09-10T18:26:22.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 -- 172.21.15.100:0/592272129 wait complete. 2024-09-10T18:26:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 Processor -- start 2024-09-10T18:26:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.915+0000 7f7288f58640 1 -- start start 2024-09-10T18:26:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7284066c90 0x7f72841133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72840fd710 0x7f72841138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:22.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72841002c0 0x7f728411a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728411aec0 con 0x7f72840fd710 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728411b030 con 0x7f7284066c90 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728411b1f0 con 0x7f72841002c0 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7281d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72840fd710 0x7f72841138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7282d9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72841002c0 0x7f728411a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7281d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72840fd710 0x7f72841138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52194/0 (socket says 172.21.15.100:52194) 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7281d9b640 1 -- 172.21.15.100:0/176430110 learned_addr learned my addr 172.21.15.100:0/176430110 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f728259c640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7284066c90 0x7f72841133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7282d9d640 1 -- 172.21.15.100:0/176430110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7284066c90 msgr2=0x7f72841133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7282d9d640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7284066c90 0x7f72841133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7282d9d640 1 -- 172.21.15.100:0/176430110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72840fd710 msgr2=0x7f72841138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7282d9d640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72840fd710 0x7f72841138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7282d9d640 1 -- 172.21.15.100:0/176430110 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c0098b0 con 0x7f72841002c0 2024-09-10T18:26:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7282d9d640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72841002c0 0x7f728411a8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f72740027d0 tx=0x7f7274009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:26:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f727affd640 1 -- 172.21.15.100:0/176430110 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274002dc0 con 0x7f72841002c0 2024-09-10T18:26:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f727affd640 1 -- 172.21.15.100:0/176430110 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f727400f040 con 0x7f72841002c0 2024-09-10T18:26:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f727affd640 1 -- 172.21.15.100:0/176430110 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274009340 con 0x7f72841002c0 2024-09-10T18:26:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c005040 con 0x7f72841002c0 2024-09-10T18:26:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.919+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728411b840 con 0x7f72841002c0 2024-09-10T18:26:22.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.923+0000 7f727affd640 1 -- 172.21.15.100:0/176430110 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 14) v1 ==== 90690+0+0 (secure 0 0 0) 0x7f7274009770 con 0x7f72841002c0 2024-09-10T18:26:22.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.923+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728404bf60 con 0x7f72841002c0 2024-09-10T18:26:22.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.923+0000 7f727affd640 1 --2- 172.21.15.100:0/176430110 >> v2:172.21.15.100:6800/3986579393 conn(0x7f727006edf0 0x7f72700712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:26:22.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.923+0000 7f728259c640 1 --2- 172.21.15.100:0/176430110 >> v2:172.21.15.100:6800/3986579393 conn(0x7f727006edf0 0x7f72700712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:26:22.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.923+0000 7f727affd640 1 -- 172.21.15.100:0/176430110 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f727408cb30 con 0x7f72841002c0 2024-09-10T18:26:22.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.923+0000 7f728259c640 1 --2- 172.21.15.100:0/176430110 >> v2:172.21.15.100:6800/3986579393 conn(0x7f727006edf0 0x7f72700712a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f726c004920 tx=0x7f726c009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:26:22.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:22.927+0000 7f727affd640 1 -- 172.21.15.100:0/176430110 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f727405aef0 con 0x7f72841002c0 2024-09-10T18:26:23.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:23.111+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f72840ff2a0 con 0x7f727006edf0 2024-09-10T18:26:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:23 smithi100 bash[18597]: audit 2024-09-10T18:26:23.122055+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:26:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:23 smithi100 bash[18597]: audit 2024-09-10T18:26:23.127862+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:26:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:23 smithi100 bash[18597]: audit 2024-09-10T18:26:23.129302+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:23 smithi100 bash[31607]: audit 2024-09-10T18:26:23.122055+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:26:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:23 smithi100 bash[31607]: audit 2024-09-10T18:26:23.127862+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:26:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:23 smithi100 bash[31607]: audit 2024-09-10T18:26:23.129302+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:23.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:23 smithi151 bash[22565]: audit 2024-09-10T18:26:23.122055+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:26:23.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:23 smithi151 bash[22565]: audit 2024-09-10T18:26:23.127862+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:26:23.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:23 smithi151 bash[22565]: audit 2024-09-10T18:26:23.129302+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:24 smithi100 bash[31607]: audit 2024-09-10T18:26:23.115465+0000 mgr.y (mgr.14150) 93 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:26:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:24 smithi100 bash[31607]: cluster 2024-09-10T18:26:23.160617+0000 mgr.y (mgr.14150) 94 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:24 smithi100 bash[18597]: audit 2024-09-10T18:26:23.115465+0000 mgr.y (mgr.14150) 93 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:26:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:24 smithi100 bash[18597]: cluster 2024-09-10T18:26:23.160617+0000 mgr.y (mgr.14150) 94 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:24.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:24 smithi151 bash[22565]: audit 2024-09-10T18:26:23.115465+0000 mgr.y (mgr.14150) 93 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:26:24.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:24 smithi151 bash[22565]: cluster 2024-09-10T18:26:23.160617+0000 mgr.y (mgr.14150) 94 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:26 smithi100 bash[18597]: cluster 2024-09-10T18:26:25.161079+0000 mgr.y (mgr.14150) 95 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:26.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:26 smithi100 bash[31607]: cluster 2024-09-10T18:26:25.161079+0000 mgr.y (mgr.14150) 95 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:26.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:26 smithi151 bash[22565]: cluster 2024-09-10T18:26:25.161079+0000 mgr.y (mgr.14150) 95 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:28.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:28 smithi100 bash[18597]: cluster 2024-09-10T18:26:27.161572+0000 mgr.y (mgr.14150) 96 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:28.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:28 smithi100 bash[31607]: cluster 2024-09-10T18:26:27.161572+0000 mgr.y (mgr.14150) 96 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:28.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:28 smithi151 bash[22565]: cluster 2024-09-10T18:26:27.161572+0000 mgr.y (mgr.14150) 96 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:30.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:30 smithi100 bash[18597]: cluster 2024-09-10T18:26:29.162014+0000 mgr.y (mgr.14150) 97 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:30 smithi100 bash[31607]: cluster 2024-09-10T18:26:29.162014+0000 mgr.y (mgr.14150) 97 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:30.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:30 smithi151 bash[22565]: cluster 2024-09-10T18:26:29.162014+0000 mgr.y (mgr.14150) 97 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[18597]: cluster 2024-09-10T18:26:31.162499+0000 mgr.y (mgr.14150) 98 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[18597]: audit 2024-09-10T18:26:31.804632+0000 mon.c (mon.2) 3 : audit [INF] from='client.? 172.21.15.100:0/492916797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]: dispatch 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[18597]: audit 2024-09-10T18:26:31.805325+0000 mon.a (mon.0) 261 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]: dispatch 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[18597]: audit 2024-09-10T18:26:31.812713+0000 mon.a (mon.0) 262 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]': finished 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[18597]: cluster 2024-09-10T18:26:31.812901+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[18597]: audit 2024-09-10T18:26:31.813054+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[31607]: cluster 2024-09-10T18:26:31.162499+0000 mgr.y (mgr.14150) 98 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[31607]: audit 2024-09-10T18:26:31.804632+0000 mon.c (mon.2) 3 : audit [INF] from='client.? 172.21.15.100:0/492916797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]: dispatch 2024-09-10T18:26:32.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[31607]: audit 2024-09-10T18:26:31.805325+0000 mon.a (mon.0) 261 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]: dispatch 2024-09-10T18:26:32.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[31607]: audit 2024-09-10T18:26:31.812713+0000 mon.a (mon.0) 262 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]': finished 2024-09-10T18:26:32.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[31607]: cluster 2024-09-10T18:26:31.812901+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:26:32.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:32 smithi100 bash[31607]: audit 2024-09-10T18:26:31.813054+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:32.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:32 smithi151 bash[22565]: cluster 2024-09-10T18:26:31.162499+0000 mgr.y (mgr.14150) 98 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:32.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:32 smithi151 bash[22565]: audit 2024-09-10T18:26:31.804632+0000 mon.c (mon.2) 3 : audit [INF] from='client.? 172.21.15.100:0/492916797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]: dispatch 2024-09-10T18:26:32.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:32 smithi151 bash[22565]: audit 2024-09-10T18:26:31.805325+0000 mon.a (mon.0) 261 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]: dispatch 2024-09-10T18:26:32.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:32 smithi151 bash[22565]: audit 2024-09-10T18:26:31.812713+0000 mon.a (mon.0) 262 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a69cb5a-7a42-4fb4-861d-55a459379aa6"}]': finished 2024-09-10T18:26:32.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:32 smithi151 bash[22565]: cluster 2024-09-10T18:26:31.812901+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:26:32.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:32 smithi151 bash[22565]: audit 2024-09-10T18:26:31.813054+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:33 smithi100 bash[18597]: audit 2024-09-10T18:26:32.440036+0000 mon.c (mon.2) 4 : audit [DBG] from='client.? 172.21.15.100:0/2206037633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:26:33.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:33 smithi100 bash[31607]: audit 2024-09-10T18:26:32.440036+0000 mon.c (mon.2) 4 : audit [DBG] from='client.? 172.21.15.100:0/2206037633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:26:33.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:33 smithi151 bash[22565]: audit 2024-09-10T18:26:32.440036+0000 mon.c (mon.2) 4 : audit [DBG] from='client.? 172.21.15.100:0/2206037633' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:26:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:34 smithi100 bash[18597]: cluster 2024-09-10T18:26:33.162981+0000 mgr.y (mgr.14150) 99 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:34 smithi100 bash[31607]: cluster 2024-09-10T18:26:33.162981+0000 mgr.y (mgr.14150) 99 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:34.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:34 smithi151 bash[22565]: cluster 2024-09-10T18:26:33.162981+0000 mgr.y (mgr.14150) 99 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:36 smithi100 bash[18597]: cluster 2024-09-10T18:26:35.163421+0000 mgr.y (mgr.14150) 100 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:36 smithi100 bash[31607]: cluster 2024-09-10T18:26:35.163421+0000 mgr.y (mgr.14150) 100 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:36.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:36 smithi151 bash[22565]: cluster 2024-09-10T18:26:35.163421+0000 mgr.y (mgr.14150) 100 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:38 smithi100 bash[18597]: cluster 2024-09-10T18:26:37.163849+0000 mgr.y (mgr.14150) 101 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:38.566 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:38 smithi100 bash[31607]: cluster 2024-09-10T18:26:37.163849+0000 mgr.y (mgr.14150) 101 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:38.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:38 smithi151 bash[22565]: cluster 2024-09-10T18:26:37.163849+0000 mgr.y (mgr.14150) 101 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:40 smithi100 bash[18597]: cluster 2024-09-10T18:26:39.164280+0000 mgr.y (mgr.14150) 102 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:40 smithi100 bash[31607]: cluster 2024-09-10T18:26:39.164280+0000 mgr.y (mgr.14150) 102 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:40.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:40 smithi151 bash[22565]: cluster 2024-09-10T18:26:39.164280+0000 mgr.y (mgr.14150) 102 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:42.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:42 smithi100 bash[18597]: cluster 2024-09-10T18:26:41.164754+0000 mgr.y (mgr.14150) 103 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:42.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:42 smithi100 bash[31607]: cluster 2024-09-10T18:26:41.164754+0000 mgr.y (mgr.14150) 103 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:42.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:42 smithi151 bash[22565]: cluster 2024-09-10T18:26:41.164754+0000 mgr.y (mgr.14150) 103 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:44 smithi100 bash[18597]: cluster 2024-09-10T18:26:43.165230+0000 mgr.y (mgr.14150) 104 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:44 smithi100 bash[31607]: cluster 2024-09-10T18:26:43.165230+0000 mgr.y (mgr.14150) 104 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:44.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:44 smithi151 bash[22565]: cluster 2024-09-10T18:26:43.165230+0000 mgr.y (mgr.14150) 104 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:45 smithi100 bash[18597]: audit 2024-09-10T18:26:44.423897+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:45 smithi100 bash[18597]: audit 2024-09-10T18:26:44.425512+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:45 smithi100 bash[18597]: cephadm 2024-09-10T18:26:44.426755+0000 mgr.y (mgr.14150) 105 : cephadm [INF] Deploying daemon osd.0 on smithi100 2024-09-10T18:26:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:45 smithi100 bash[31607]: audit 2024-09-10T18:26:44.423897+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:45.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:45 smithi100 bash[31607]: audit 2024-09-10T18:26:44.425512+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:45.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:45 smithi100 bash[31607]: cephadm 2024-09-10T18:26:44.426755+0000 mgr.y (mgr.14150) 105 : cephadm [INF] Deploying daemon osd.0 on smithi100 2024-09-10T18:26:45.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:45 smithi151 bash[22565]: audit 2024-09-10T18:26:44.423897+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:26:45.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:45 smithi151 bash[22565]: audit 2024-09-10T18:26:44.425512+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:45.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:45 smithi151 bash[22565]: cephadm 2024-09-10T18:26:44.426755+0000 mgr.y (mgr.14150) 105 : cephadm [INF] Deploying daemon osd.0 on smithi100 2024-09-10T18:26:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:46 smithi100 bash[31607]: cluster 2024-09-10T18:26:45.165736+0000 mgr.y (mgr.14150) 106 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:46 smithi100 bash[18597]: cluster 2024-09-10T18:26:45.165736+0000 mgr.y (mgr.14150) 106 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:46.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:46 smithi151 bash[22565]: cluster 2024-09-10T18:26:45.165736+0000 mgr.y (mgr.14150) 106 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:48 smithi100 bash[18597]: cluster 2024-09-10T18:26:47.166202+0000 mgr.y (mgr.14150) 107 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:48 smithi100 bash[31607]: cluster 2024-09-10T18:26:47.166202+0000 mgr.y (mgr.14150) 107 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:48.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:48 smithi151 bash[22565]: cluster 2024-09-10T18:26:47.166202+0000 mgr.y (mgr.14150) 107 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:49 smithi100 bash[18597]: audit 2024-09-10T18:26:49.052998+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:49 smithi100 bash[18597]: audit 2024-09-10T18:26:49.057203+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:49 smithi100 bash[18597]: audit 2024-09-10T18:26:49.062160+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:49 smithi100 bash[31607]: audit 2024-09-10T18:26:49.052998+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:49 smithi100 bash[31607]: audit 2024-09-10T18:26:49.057203+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:49 smithi100 bash[31607]: audit 2024-09-10T18:26:49.062160+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:49.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:49 smithi151 bash[22565]: audit 2024-09-10T18:26:49.052998+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:49.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:49 smithi151 bash[22565]: audit 2024-09-10T18:26:49.057203+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:49.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:49 smithi151 bash[22565]: audit 2024-09-10T18:26:49.062160+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:50.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:50 smithi100 bash[18597]: cluster 2024-09-10T18:26:49.166511+0000 mgr.y (mgr.14150) 108 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:50.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:50 smithi100 bash[31607]: cluster 2024-09-10T18:26:49.166511+0000 mgr.y (mgr.14150) 108 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:50.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:50 smithi151 bash[22565]: cluster 2024-09-10T18:26:49.166511+0000 mgr.y (mgr.14150) 108 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:52.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:52 smithi151 bash[22565]: cluster 2024-09-10T18:26:51.166972+0000 mgr.y (mgr.14150) 109 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:52 smithi100 bash[18597]: cluster 2024-09-10T18:26:51.166972+0000 mgr.y (mgr.14150) 109 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:52.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:52 smithi100 bash[31607]: cluster 2024-09-10T18:26:51.166972+0000 mgr.y (mgr.14150) 109 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:54.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:54 smithi151 bash[22565]: cluster 2024-09-10T18:26:53.167454+0000 mgr.y (mgr.14150) 110 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:54 smithi100 bash[18597]: cluster 2024-09-10T18:26:53.167454+0000 mgr.y (mgr.14150) 110 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:54 smithi100 bash[31607]: cluster 2024-09-10T18:26:53.167454+0000 mgr.y (mgr.14150) 110 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: cluster 2024-09-10T18:26:55.167916+0000 mgr.y (mgr.14150) 111 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: audit 2024-09-10T18:26:55.653003+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: audit 2024-09-10T18:26:55.659791+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: audit 2024-09-10T18:26:56.050373+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: audit 2024-09-10T18:26:56.051613+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: audit 2024-09-10T18:26:56.059329+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: audit 2024-09-10T18:26:56.061282+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 v2:172.21.15.100:6801/2899184110' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[18597]: audit 2024-09-10T18:26:56.061672+0000 mon.a (mon.0) 275 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: cluster 2024-09-10T18:26:55.167916+0000 mgr.y (mgr.14150) 111 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: audit 2024-09-10T18:26:55.653003+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: audit 2024-09-10T18:26:55.659791+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: audit 2024-09-10T18:26:56.050373+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: audit 2024-09-10T18:26:56.051613+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: audit 2024-09-10T18:26:56.059329+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: audit 2024-09-10T18:26:56.061282+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 v2:172.21.15.100:6801/2899184110' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:26:56.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:56 smithi100 bash[31607]: audit 2024-09-10T18:26:56.061672+0000 mon.a (mon.0) 275 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:26:56.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: cluster 2024-09-10T18:26:55.167916+0000 mgr.y (mgr.14150) 111 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:56.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: audit 2024-09-10T18:26:55.653003+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: audit 2024-09-10T18:26:55.659791+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: audit 2024-09-10T18:26:56.050373+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:26:56.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: audit 2024-09-10T18:26:56.051613+0000 mon.a (mon.0) 273 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:26:56.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: audit 2024-09-10T18:26:56.059329+0000 mon.a (mon.0) 274 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:56.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: audit 2024-09-10T18:26:56.061282+0000 mon.c (mon.2) 5 : audit [INF] from='osd.0 v2:172.21.15.100:6801/2899184110' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:26:56.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:56 smithi151 bash[22565]: audit 2024-09-10T18:26:56.061672+0000 mon.a (mon.0) 275 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:26:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[31607]: audit 2024-09-10T18:26:57.064789+0000 mon.a (mon.0) 276 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:26:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[31607]: cluster 2024-09-10T18:26:57.064984+0000 mon.a (mon.0) 277 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:26:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[31607]: audit 2024-09-10T18:26:57.065246+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[31607]: audit 2024-09-10T18:26:57.066055+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 v2:172.21.15.100:6801/2899184110' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:26:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[31607]: audit 2024-09-10T18:26:57.066699+0000 mon.a (mon.0) 279 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:26:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[31607]: cluster 2024-09-10T18:26:57.168371+0000 mgr.y (mgr.14150) 112 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[18597]: audit 2024-09-10T18:26:57.064789+0000 mon.a (mon.0) 276 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:26:58.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[18597]: cluster 2024-09-10T18:26:57.064984+0000 mon.a (mon.0) 277 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:26:58.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[18597]: audit 2024-09-10T18:26:57.065246+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:58.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[18597]: audit 2024-09-10T18:26:57.066055+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 v2:172.21.15.100:6801/2899184110' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:26:58.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[18597]: audit 2024-09-10T18:26:57.066699+0000 mon.a (mon.0) 279 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:26:58.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:58 smithi100 bash[18597]: cluster 2024-09-10T18:26:57.168371+0000 mgr.y (mgr.14150) 112 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:58.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:58 smithi151 bash[22565]: audit 2024-09-10T18:26:57.064789+0000 mon.a (mon.0) 276 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:26:58.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:58 smithi151 bash[22565]: cluster 2024-09-10T18:26:57.064984+0000 mon.a (mon.0) 277 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:26:58.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:58 smithi151 bash[22565]: audit 2024-09-10T18:26:57.065246+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:58.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:58 smithi151 bash[22565]: audit 2024-09-10T18:26:57.066055+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 v2:172.21.15.100:6801/2899184110' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:26:58.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:58 smithi151 bash[22565]: audit 2024-09-10T18:26:57.066699+0000 mon.a (mon.0) 279 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:26:58.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:58 smithi151 bash[22565]: cluster 2024-09-10T18:26:57.168371+0000 mgr.y (mgr.14150) 112 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:26:59.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.058+0000 7f727affd640 1 -- 172.21.15.100:0/176430110 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f72840ff2a0 con 0x7f727006edf0 2024-09-10T18:26:59.076 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 0 on host 'smithi100' 2024-09-10T18:26:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 >> v2:172.21.15.100:6800/3986579393 conn(0x7f727006edf0 msgr2=0x7f72700712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 --2- 172.21.15.100:0/176430110 >> v2:172.21.15.100:6800/3986579393 conn(0x7f727006edf0 0x7f72700712a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f726c004920 tx=0x7f726c009da0 comp rx=0 tx=0).stop 2024-09-10T18:26:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72841002c0 msgr2=0x7f728411a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:26:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72841002c0 0x7f728411a8f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f72740027d0 tx=0x7f7274009f70 comp rx=0 tx=0).stop 2024-09-10T18:26:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 shutdown_connections 2024-09-10T18:26:59.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 --2- 172.21.15.100:0/176430110 >> v2:172.21.15.100:6800/3986579393 conn(0x7f727006edf0 0x7f72700712a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:59.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72841002c0 0x7f728411a8f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:59.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72840fd710 0x7f72841138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:59.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 --2- 172.21.15.100:0/176430110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7284066c90 0x7f72841133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:26:59.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 >> 172.21.15.100:0/176430110 conn(0x7f72840f6e90 msgr2=0x7f72840fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:26:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 shutdown_connections 2024-09-10T18:26:59.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:26:59.062+0000 7f7288f58640 1 -- 172.21.15.100:0/176430110 wait complete. 2024-09-10T18:26:59.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: cluster 2024-09-10T18:26:57.052477+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: cluster 2024-09-10T18:26:57.052586+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: audit 2024-09-10T18:26:58.065195+0000 mon.a (mon.0) 280 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: cluster 2024-09-10T18:26:58.065467+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: audit 2024-09-10T18:26:58.066268+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: audit 2024-09-10T18:26:58.074303+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: audit 2024-09-10T18:26:59.046756+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: audit 2024-09-10T18:26:59.051672+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:59.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:26:59 smithi151 bash[22565]: audit 2024-09-10T18:26:59.059691+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: cluster 2024-09-10T18:26:57.052477+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:26:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: cluster 2024-09-10T18:26:57.052586+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:26:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: audit 2024-09-10T18:26:58.065195+0000 mon.a (mon.0) 280 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:26:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: cluster 2024-09-10T18:26:58.065467+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:26:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: audit 2024-09-10T18:26:58.066268+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: audit 2024-09-10T18:26:58.074303+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: audit 2024-09-10T18:26:59.046756+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: audit 2024-09-10T18:26:59.051672+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[18597]: audit 2024-09-10T18:26:59.059691+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: cluster 2024-09-10T18:26:57.052477+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: cluster 2024-09-10T18:26:57.052586+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: audit 2024-09-10T18:26:58.065195+0000 mon.a (mon.0) 280 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: cluster 2024-09-10T18:26:58.065467+0000 mon.a (mon.0) 281 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: audit 2024-09-10T18:26:58.066268+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: audit 2024-09-10T18:26:58.074303+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: audit 2024-09-10T18:26:59.046756+0000 mon.a (mon.0) 284 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: audit 2024-09-10T18:26:59.051672+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:26:59 smithi100 bash[31607]: audit 2024-09-10T18:26:59.059691+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:26:59.713 DEBUG:teuthology.orchestra.run.smithi100:osd.0> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:26:59.716 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_3... 2024-09-10T18:26:59.716 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:26:59.855 INFO:journalctl@ceph.osd.0.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:27:00.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:00 smithi151 bash[22565]: audit 2024-09-10T18:26:59.076838+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:27:00.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:00 smithi151 bash[22565]: cluster 2024-09-10T18:26:59.081479+0000 mon.a (mon.0) 288 : cluster [INF] osd.0 v2:172.21.15.100:6801/2899184110 boot 2024-09-10T18:27:00.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:00 smithi151 bash[22565]: cluster 2024-09-10T18:26:59.081534+0000 mon.a (mon.0) 289 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:27:00.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:00 smithi151 bash[22565]: audit 2024-09-10T18:26:59.081700+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:27:00.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:00 smithi151 bash[22565]: cluster 2024-09-10T18:26:59.168745+0000 mgr.y (mgr.14150) 113 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:27:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[31607]: audit 2024-09-10T18:26:59.076838+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:27:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[31607]: cluster 2024-09-10T18:26:59.081479+0000 mon.a (mon.0) 288 : cluster [INF] osd.0 v2:172.21.15.100:6801/2899184110 boot 2024-09-10T18:27:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[31607]: cluster 2024-09-10T18:26:59.081534+0000 mon.a (mon.0) 289 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:27:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[31607]: audit 2024-09-10T18:26:59.081700+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:27:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[31607]: cluster 2024-09-10T18:26:59.168745+0000 mgr.y (mgr.14150) 113 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:27:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[18597]: audit 2024-09-10T18:26:59.076838+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:27:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[18597]: cluster 2024-09-10T18:26:59.081479+0000 mon.a (mon.0) 288 : cluster [INF] osd.0 v2:172.21.15.100:6801/2899184110 boot 2024-09-10T18:27:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[18597]: cluster 2024-09-10T18:26:59.081534+0000 mon.a (mon.0) 289 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:27:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[18597]: audit 2024-09-10T18:26:59.081700+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:27:00.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:00 smithi100 bash[18597]: cluster 2024-09-10T18:26:59.168745+0000 mgr.y (mgr.14150) 113 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:27:01.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:01 smithi151 bash[22565]: cluster 2024-09-10T18:27:00.083903+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:27:01.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:01 smithi100 bash[18597]: cluster 2024-09-10T18:27:00.083903+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:27:01.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:01 smithi100 bash[31607]: cluster 2024-09-10T18:27:00.083903+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:27:02.416 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:02 smithi151 bash[22565]: cluster 2024-09-10T18:27:01.169236+0000 mgr.y (mgr.14150) 114 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:02 smithi100 bash[18597]: cluster 2024-09-10T18:27:01.169236+0000 mgr.y (mgr.14150) 114 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:02 smithi100 bash[31607]: cluster 2024-09-10T18:27:01.169236+0000 mgr.y (mgr.14150) 114 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:27:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:04 smithi100 bash[18597]: cluster 2024-09-10T18:27:03.169738+0000 mgr.y (mgr.14150) 115 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:04 smithi100 bash[31607]: cluster 2024-09-10T18:27:03.169738+0000 mgr.y (mgr.14150) 115 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:04.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:04 smithi151 bash[22565]: cluster 2024-09-10T18:27:03.169738+0000 mgr.y (mgr.14150) 115 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:06 smithi100 bash[18597]: cluster 2024-09-10T18:27:05.170202+0000 mgr.y (mgr.14150) 116 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:06.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:06 smithi100 bash[31607]: cluster 2024-09-10T18:27:05.170202+0000 mgr.y (mgr.14150) 116 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:06.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:06 smithi151 bash[22565]: cluster 2024-09-10T18:27:05.170202+0000 mgr.y (mgr.14150) 116 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:08.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:08 smithi100 bash[18597]: cluster 2024-09-10T18:27:07.170713+0000 mgr.y (mgr.14150) 117 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:08.491 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:08 smithi100 bash[31607]: cluster 2024-09-10T18:27:07.170713+0000 mgr.y (mgr.14150) 117 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:08.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:08 smithi151 bash[22565]: cluster 2024-09-10T18:27:07.170713+0000 mgr.y (mgr.14150) 117 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:08.838 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:27:09.013 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:27:09.034 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-10T18:27:09.034 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-10T18:27:09.035 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0179301 s, 585 MB/s 2024-09-10T18:27:09.035 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-10T18:27:09.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-09-10T18:27:10.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:10 smithi100 bash[18597]: cluster 2024-09-10T18:27:09.171111+0000 mgr.y (mgr.14150) 118 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:10.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:10 smithi100 bash[31607]: cluster 2024-09-10T18:27:09.171111+0000 mgr.y (mgr.14150) 118 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:10.667 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:10 smithi151 bash[22565]: cluster 2024-09-10T18:27:09.171111+0000 mgr.y (mgr.14150) 118 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: cephadm 2024-09-10T18:27:10.495202+0000 mgr.y (mgr.14150) 119 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:27:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: audit 2024-09-10T18:27:10.503263+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: audit 2024-09-10T18:27:10.510814+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: audit 2024-09-10T18:27:10.512392+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:27:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: cephadm 2024-09-10T18:27:10.513484+0000 mgr.y (mgr.14150) 120 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 16246M 2024-09-10T18:27:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: audit 2024-09-10T18:27:10.521755+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: audit 2024-09-10T18:27:10.523993+0000 mon.a (mon.0) 296 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: audit 2024-09-10T18:27:10.525370+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[18597]: audit 2024-09-10T18:27:10.536498+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: cephadm 2024-09-10T18:27:10.495202+0000 mgr.y (mgr.14150) 119 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: audit 2024-09-10T18:27:10.503263+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: audit 2024-09-10T18:27:10.510814+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: audit 2024-09-10T18:27:10.512392+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: cephadm 2024-09-10T18:27:10.513484+0000 mgr.y (mgr.14150) 120 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 16246M 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: audit 2024-09-10T18:27:10.521755+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: audit 2024-09-10T18:27:10.523993+0000 mon.a (mon.0) 296 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: audit 2024-09-10T18:27:10.525370+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:11 smithi100 bash[31607]: audit 2024-09-10T18:27:10.536498+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: cephadm 2024-09-10T18:27:10.495202+0000 mgr.y (mgr.14150) 119 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: audit 2024-09-10T18:27:10.503263+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: audit 2024-09-10T18:27:10.510814+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: audit 2024-09-10T18:27:10.512392+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: cephadm 2024-09-10T18:27:10.513484+0000 mgr.y (mgr.14150) 120 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 16246M 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: audit 2024-09-10T18:27:10.521755+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: audit 2024-09-10T18:27:10.523993+0000 mon.a (mon.0) 296 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:11.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: audit 2024-09-10T18:27:10.525370+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:11.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:11 smithi151 bash[22565]: audit 2024-09-10T18:27:10.536498+0000 mon.a (mon.0) 298 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:12 smithi100 bash[18597]: cluster 2024-09-10T18:27:11.171626+0000 mgr.y (mgr.14150) 121 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:12 smithi100 bash[31607]: cluster 2024-09-10T18:27:11.171626+0000 mgr.y (mgr.14150) 121 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:12.916 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:12 smithi151 bash[22565]: cluster 2024-09-10T18:27:11.171626+0000 mgr.y (mgr.14150) 121 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:14.447 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:27:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:14 smithi100 bash[18597]: cluster 2024-09-10T18:27:13.172132+0000 mgr.y (mgr.14150) 122 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:14.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:14 smithi100 bash[31607]: cluster 2024-09-10T18:27:13.172132+0000 mgr.y (mgr.14150) 122 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:14.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:14 smithi151 bash[22565]: cluster 2024-09-10T18:27:13.172132+0000 mgr.y (mgr.14150) 122 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:16 smithi100 bash[31607]: cluster 2024-09-10T18:27:15.172644+0000 mgr.y (mgr.14150) 123 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:16 smithi100 bash[18597]: cluster 2024-09-10T18:27:15.172644+0000 mgr.y (mgr.14150) 123 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:16.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:16 smithi151 bash[22565]: cluster 2024-09-10T18:27:15.172644+0000 mgr.y (mgr.14150) 123 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- 172.21.15.100:0/1066751869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10066c70 msgr2=0x7f8d100ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 --2- 172.21.15.100:0/1066751869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10066c70 0x7f8d100ff480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c004950 tx=0x7f8d0c009da0 comp rx=0 tx=0).stop 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- 172.21.15.100:0/1066751869 shutdown_connections 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 --2- 172.21.15.100:0/1066751869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d101024d0 0x7f8d10104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 --2- 172.21.15.100:0/1066751869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100ff9c0 0x7f8d10101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 --2- 172.21.15.100:0/1066751869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10066c70 0x7f8d100ff480 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- 172.21.15.100:0/1066751869 >> 172.21.15.100:0/1066751869 conn(0x7f8d100f6f30 msgr2=0x7f8d100f9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- 172.21.15.100:0/1066751869 shutdown_connections 2024-09-10T18:27:17.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- 172.21.15.100:0/1066751869 wait complete. 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 Processor -- start 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- start start 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10066c70 0x7f8d100fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d100ff9c0 0x7f8d100fb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d101024d0 0x7f8d100fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d100fbf60 con 0x7f8d100ff9c0 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d100fc0d0 con 0x7f8d10066c70 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d10116f10 con 0x7f8d101024d0 2024-09-10T18:27:17.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d16c62640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d101024d0 0x7f8d100fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d100ff9c0 0x7f8d100fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d16c62640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d101024d0 0x7f8d100fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:59832/0 (socket says 172.21.15.100:59832) 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d100ff9c0 0x7f8d100fb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37042/0 (socket says 172.21.15.100:37042) 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d16c62640 1 -- 172.21.15.100:0/261661554 learned_addr learned my addr 172.21.15.100:0/261661554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d16461640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10066c70 0x7f8d100fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 -- 172.21.15.100:0/261661554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d101024d0 msgr2=0x7f8d100fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d101024d0 0x7f8d100fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 -- 172.21.15.100:0/261661554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10066c70 msgr2=0x7f8d100fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10066c70 0x7f8d100fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 -- 172.21.15.100:0/261661554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c005040 con 0x7f8d100ff9c0 2024-09-10T18:27:17.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d15c60640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d100ff9c0 0x7f8d100fb380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ebc0 tx=0x7f8d0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:27:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d02ffd640 1 -- 172.21.15.100:0/261661554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04007c30 con 0x7f8d100ff9c0 2024-09-10T18:27:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d02ffd640 1 -- 172.21.15.100:0/261661554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8d0400f040 con 0x7f8d100ff9c0 2024-09-10T18:27:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d101170b0 con 0x7f8d100ff9c0 2024-09-10T18:27:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d02ffd640 1 -- 172.21.15.100:0/261661554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04009300 con 0x7f8d100ff9c0 2024-09-10T18:27:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.694+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d10117580 con 0x7f8d100ff9c0 2024-09-10T18:27:17.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.698+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1004bf60 con 0x7f8d100ff9c0 2024-09-10T18:27:17.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.698+0000 7f8d02ffd640 1 -- 172.21.15.100:0/261661554 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 14) v1 ==== 90690+0+0 (secure 0 0 0) 0x7f8d04009730 con 0x7f8d100ff9c0 2024-09-10T18:27:17.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.698+0000 7f8d02ffd640 1 --2- 172.21.15.100:0/261661554 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8cfc06ed40 0x7f8cfc0711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:27:17.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.698+0000 7f8d02ffd640 1 -- 172.21.15.100:0/261661554 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1537+0+0 (secure 0 0 0) 0x7f8d0408cec0 con 0x7f8d100ff9c0 2024-09-10T18:27:17.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.698+0000 7f8d16461640 1 --2- 172.21.15.100:0/261661554 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8cfc06ed40 0x7f8cfc0711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:27:17.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.702+0000 7f8d16461640 1 --2- 172.21.15.100:0/261661554 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8cfc06ed40 0x7f8cfc0711f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c004920 tx=0x7f8d0c009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:27:17.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.706+0000 7f8d02ffd640 1 -- 172.21.15.100:0/261661554 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8d0405b200 con 0x7f8d100ff9c0 2024-09-10T18:27:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:17.890+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d10104060 con 0x7f8cfc06ed40 2024-09-10T18:27:18.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:18 smithi151 bash[22565]: cluster 2024-09-10T18:27:17.173122+0000 mgr.y (mgr.14150) 124 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:18.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:18 smithi151 bash[22565]: audit 2024-09-10T18:27:17.902568+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:27:18.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:18 smithi151 bash[22565]: audit 2024-09-10T18:27:17.906597+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:27:18.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:18 smithi151 bash[22565]: audit 2024-09-10T18:27:17.908039+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[18597]: cluster 2024-09-10T18:27:17.173122+0000 mgr.y (mgr.14150) 124 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[18597]: audit 2024-09-10T18:27:17.902568+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:27:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[18597]: audit 2024-09-10T18:27:17.906597+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:27:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[18597]: audit 2024-09-10T18:27:17.908039+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[31607]: cluster 2024-09-10T18:27:17.173122+0000 mgr.y (mgr.14150) 124 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[31607]: audit 2024-09-10T18:27:17.902568+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:27:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[31607]: audit 2024-09-10T18:27:17.906597+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:27:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:18 smithi100 bash[31607]: audit 2024-09-10T18:27:17.908039+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:19.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:19 smithi151 bash[22565]: audit 2024-09-10T18:27:17.896545+0000 mgr.y (mgr.14150) 125 : audit [DBG] from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:19 smithi100 bash[18597]: audit 2024-09-10T18:27:17.896545+0000 mgr.y (mgr.14150) 125 : audit [DBG] from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:19 smithi100 bash[31607]: audit 2024-09-10T18:27:17.896545+0000 mgr.y (mgr.14150) 125 : audit [DBG] from='client.14241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:27:20.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:20 smithi151 bash[22565]: cluster 2024-09-10T18:27:19.173613+0000 mgr.y (mgr.14150) 126 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:20 smithi100 bash[18597]: cluster 2024-09-10T18:27:19.173613+0000 mgr.y (mgr.14150) 126 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:20 smithi100 bash[31607]: cluster 2024-09-10T18:27:19.173613+0000 mgr.y (mgr.14150) 126 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:22.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:22 smithi151 bash[22565]: cluster 2024-09-10T18:27:21.174075+0000 mgr.y (mgr.14150) 127 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:22 smithi100 bash[18597]: cluster 2024-09-10T18:27:21.174075+0000 mgr.y (mgr.14150) 127 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:23.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:22 smithi100 bash[31607]: cluster 2024-09-10T18:27:21.174075+0000 mgr.y (mgr.14150) 127 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:24.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:24 smithi151 bash[22565]: cluster 2024-09-10T18:27:23.174541+0000 mgr.y (mgr.14150) 128 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:25.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:24 smithi100 bash[18597]: cluster 2024-09-10T18:27:23.174541+0000 mgr.y (mgr.14150) 128 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:24 smithi100 bash[31607]: cluster 2024-09-10T18:27:23.174541+0000 mgr.y (mgr.14150) 128 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:26.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:26 smithi151 bash[22565]: cluster 2024-09-10T18:27:25.175010+0000 mgr.y (mgr.14150) 129 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:26.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:26 smithi151 bash[22565]: audit 2024-09-10T18:27:26.455709+0000 mon.a (mon.0) 302 : audit [INF] from='client.? 172.21.15.100:0/3823609327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ad7728-dc92-4291-99cd-18ebfacd6c4e"}]: dispatch 2024-09-10T18:27:26.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:26 smithi151 bash[22565]: audit 2024-09-10T18:27:26.465562+0000 mon.a (mon.0) 303 : audit [INF] from='client.? 172.21.15.100:0/3823609327' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ad7728-dc92-4291-99cd-18ebfacd6c4e"}]': finished 2024-09-10T18:27:26.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:26 smithi151 bash[22565]: cluster 2024-09-10T18:27:26.465728+0000 mon.a (mon.0) 304 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:27:26.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:26 smithi151 bash[22565]: audit 2024-09-10T18:27:26.465899+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[31607]: cluster 2024-09-10T18:27:25.175010+0000 mgr.y (mgr.14150) 129 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[31607]: audit 2024-09-10T18:27:26.455709+0000 mon.a (mon.0) 302 : audit [INF] from='client.? 172.21.15.100:0/3823609327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ad7728-dc92-4291-99cd-18ebfacd6c4e"}]: dispatch 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[31607]: audit 2024-09-10T18:27:26.465562+0000 mon.a (mon.0) 303 : audit [INF] from='client.? 172.21.15.100:0/3823609327' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ad7728-dc92-4291-99cd-18ebfacd6c4e"}]': finished 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[31607]: cluster 2024-09-10T18:27:26.465728+0000 mon.a (mon.0) 304 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[31607]: audit 2024-09-10T18:27:26.465899+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[18597]: cluster 2024-09-10T18:27:25.175010+0000 mgr.y (mgr.14150) 129 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[18597]: audit 2024-09-10T18:27:26.455709+0000 mon.a (mon.0) 302 : audit [INF] from='client.? 172.21.15.100:0/3823609327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ad7728-dc92-4291-99cd-18ebfacd6c4e"}]: dispatch 2024-09-10T18:27:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[18597]: audit 2024-09-10T18:27:26.465562+0000 mon.a (mon.0) 303 : audit [INF] from='client.? 172.21.15.100:0/3823609327' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ad7728-dc92-4291-99cd-18ebfacd6c4e"}]': finished 2024-09-10T18:27:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[18597]: cluster 2024-09-10T18:27:26.465728+0000 mon.a (mon.0) 304 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:27:27.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:26 smithi100 bash[18597]: audit 2024-09-10T18:27:26.465899+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:27.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:27 smithi151 bash[22565]: audit 2024-09-10T18:27:27.116705+0000 mon.c (mon.2) 7 : audit [DBG] from='client.? 172.21.15.100:0/3389353904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:27:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:27 smithi100 bash[18597]: audit 2024-09-10T18:27:27.116705+0000 mon.c (mon.2) 7 : audit [DBG] from='client.? 172.21.15.100:0/3389353904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:27:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:27 smithi100 bash[31607]: audit 2024-09-10T18:27:27.116705+0000 mon.c (mon.2) 7 : audit [DBG] from='client.? 172.21.15.100:0/3389353904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:27:28.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:28 smithi151 bash[22565]: cluster 2024-09-10T18:27:27.175418+0000 mgr.y (mgr.14150) 130 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:28 smithi100 bash[18597]: cluster 2024-09-10T18:27:27.175418+0000 mgr.y (mgr.14150) 130 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:28 smithi100 bash[31607]: cluster 2024-09-10T18:27:27.175418+0000 mgr.y (mgr.14150) 130 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:30.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:30 smithi151 bash[22565]: cluster 2024-09-10T18:27:29.175899+0000 mgr.y (mgr.14150) 131 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:30 smithi100 bash[18597]: cluster 2024-09-10T18:27:29.175899+0000 mgr.y (mgr.14150) 131 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:30 smithi100 bash[31607]: cluster 2024-09-10T18:27:29.175899+0000 mgr.y (mgr.14150) 131 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:32.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:32 smithi151 bash[22565]: cluster 2024-09-10T18:27:31.176382+0000 mgr.y (mgr.14150) 132 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:32 smithi100 bash[18597]: cluster 2024-09-10T18:27:31.176382+0000 mgr.y (mgr.14150) 132 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:32 smithi100 bash[31607]: cluster 2024-09-10T18:27:31.176382+0000 mgr.y (mgr.14150) 132 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:34.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:34 smithi151 bash[22565]: cluster 2024-09-10T18:27:33.176894+0000 mgr.y (mgr.14150) 133 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:34 smithi100 bash[18597]: cluster 2024-09-10T18:27:33.176894+0000 mgr.y (mgr.14150) 133 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:34 smithi100 bash[31607]: cluster 2024-09-10T18:27:33.176894+0000 mgr.y (mgr.14150) 133 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:36 smithi100 bash[18597]: cluster 2024-09-10T18:27:35.177365+0000 mgr.y (mgr.14150) 134 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:36 smithi100 bash[31607]: cluster 2024-09-10T18:27:35.177365+0000 mgr.y (mgr.14150) 134 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:36.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:36 smithi151 bash[22565]: cluster 2024-09-10T18:27:35.177365+0000 mgr.y (mgr.14150) 134 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:38 smithi100 bash[18597]: cluster 2024-09-10T18:27:37.177829+0000 mgr.y (mgr.14150) 135 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:38 smithi100 bash[31607]: cluster 2024-09-10T18:27:37.177829+0000 mgr.y (mgr.14150) 135 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:38.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:38 smithi151 bash[22565]: cluster 2024-09-10T18:27:37.177829+0000 mgr.y (mgr.14150) 135 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:39.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:39 smithi100 bash[18597]: audit 2024-09-10T18:27:39.146228+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:39.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:39 smithi100 bash[18597]: audit 2024-09-10T18:27:39.147670+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:39 smithi100 bash[31607]: audit 2024-09-10T18:27:39.146228+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:39 smithi100 bash[31607]: audit 2024-09-10T18:27:39.147670+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:39.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:39 smithi151 bash[22565]: audit 2024-09-10T18:27:39.146228+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:27:39.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:39 smithi151 bash[22565]: audit 2024-09-10T18:27:39.147670+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:40.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:40 smithi151 bash[22565]: cephadm 2024-09-10T18:27:39.148866+0000 mgr.y (mgr.14150) 136 : cephadm [INF] Deploying daemon osd.1 on smithi100 2024-09-10T18:27:40.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:40 smithi151 bash[22565]: cluster 2024-09-10T18:27:39.178226+0000 mgr.y (mgr.14150) 137 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:40 smithi100 bash[31607]: cephadm 2024-09-10T18:27:39.148866+0000 mgr.y (mgr.14150) 136 : cephadm [INF] Deploying daemon osd.1 on smithi100 2024-09-10T18:27:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:40 smithi100 bash[31607]: cluster 2024-09-10T18:27:39.178226+0000 mgr.y (mgr.14150) 137 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:40 smithi100 bash[18597]: cephadm 2024-09-10T18:27:39.148866+0000 mgr.y (mgr.14150) 136 : cephadm [INF] Deploying daemon osd.1 on smithi100 2024-09-10T18:27:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:40 smithi100 bash[18597]: cluster 2024-09-10T18:27:39.178226+0000 mgr.y (mgr.14150) 137 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:42 smithi100 bash[18597]: cluster 2024-09-10T18:27:41.178700+0000 mgr.y (mgr.14150) 138 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:42 smithi100 bash[31607]: cluster 2024-09-10T18:27:41.178700+0000 mgr.y (mgr.14150) 138 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:42.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:42 smithi151 bash[22565]: cluster 2024-09-10T18:27:41.178700+0000 mgr.y (mgr.14150) 138 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:43.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:43 smithi151 bash[22565]: audit 2024-09-10T18:27:43.574940+0000 mon.a (mon.0) 308 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:43.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:43 smithi151 bash[22565]: audit 2024-09-10T18:27:43.579771+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:43.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:43 smithi151 bash[22565]: audit 2024-09-10T18:27:43.587849+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:43 smithi100 bash[18597]: audit 2024-09-10T18:27:43.574940+0000 mon.a (mon.0) 308 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:43 smithi100 bash[18597]: audit 2024-09-10T18:27:43.579771+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:43 smithi100 bash[18597]: audit 2024-09-10T18:27:43.587849+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:43 smithi100 bash[31607]: audit 2024-09-10T18:27:43.574940+0000 mon.a (mon.0) 308 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:43 smithi100 bash[31607]: audit 2024-09-10T18:27:43.579771+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:43 smithi100 bash[31607]: audit 2024-09-10T18:27:43.587849+0000 mon.a (mon.0) 310 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:44.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:44 smithi151 bash[22565]: cluster 2024-09-10T18:27:43.179283+0000 mgr.y (mgr.14150) 139 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:45.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:44 smithi100 bash[18597]: cluster 2024-09-10T18:27:43.179283+0000 mgr.y (mgr.14150) 139 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:44 smithi100 bash[31607]: cluster 2024-09-10T18:27:43.179283+0000 mgr.y (mgr.14150) 139 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:46.895 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:46 smithi100 bash[18597]: cluster 2024-09-10T18:27:45.179729+0000 mgr.y (mgr.14150) 140 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:46.895 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:46 smithi100 bash[31607]: cluster 2024-09-10T18:27:45.179729+0000 mgr.y (mgr.14150) 140 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:46.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:46 smithi151 bash[22565]: cluster 2024-09-10T18:27:45.179729+0000 mgr.y (mgr.14150) 140 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:48.917 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:48 smithi151 bash[22565]: cluster 2024-09-10T18:27:47.180178+0000 mgr.y (mgr.14150) 141 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:49.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:48 smithi100 bash[18597]: cluster 2024-09-10T18:27:47.180178+0000 mgr.y (mgr.14150) 141 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:49.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:48 smithi100 bash[31607]: cluster 2024-09-10T18:27:47.180178+0000 mgr.y (mgr.14150) 141 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:50.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[18597]: cluster 2024-09-10T18:27:49.180666+0000 mgr.y (mgr.14150) 142 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:50.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[18597]: audit 2024-09-10T18:27:49.975402+0000 mon.a (mon.0) 311 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[18597]: audit 2024-09-10T18:27:49.986213+0000 mon.a (mon.0) 312 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[18597]: audit 2024-09-10T18:27:49.988104+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[18597]: audit 2024-09-10T18:27:49.989931+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[18597]: audit 2024-09-10T18:27:50.000330+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[18597]: audit 2024-09-10T18:27:50.482565+0000 mon.a (mon.0) 316 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[31607]: cluster 2024-09-10T18:27:49.180666+0000 mgr.y (mgr.14150) 142 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[31607]: audit 2024-09-10T18:27:49.975402+0000 mon.a (mon.0) 311 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[31607]: audit 2024-09-10T18:27:49.986213+0000 mon.a (mon.0) 312 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[31607]: audit 2024-09-10T18:27:49.988104+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[31607]: audit 2024-09-10T18:27:49.989931+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[31607]: audit 2024-09-10T18:27:50.000330+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:50 smithi100 bash[31607]: audit 2024-09-10T18:27:50.482565+0000 mon.a (mon.0) 316 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:27:50.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:50 smithi151 bash[22565]: cluster 2024-09-10T18:27:49.180666+0000 mgr.y (mgr.14150) 142 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:50.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:50 smithi151 bash[22565]: audit 2024-09-10T18:27:49.975402+0000 mon.a (mon.0) 311 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:50 smithi151 bash[22565]: audit 2024-09-10T18:27:49.986213+0000 mon.a (mon.0) 312 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:50 smithi151 bash[22565]: audit 2024-09-10T18:27:49.988104+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:27:50.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:50 smithi151 bash[22565]: audit 2024-09-10T18:27:49.989931+0000 mon.a (mon.0) 314 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:27:50.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:50 smithi151 bash[22565]: audit 2024-09-10T18:27:50.000330+0000 mon.a (mon.0) 315 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:50.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:50 smithi151 bash[22565]: audit 2024-09-10T18:27:50.482565+0000 mon.a (mon.0) 316 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:27:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[18597]: audit 2024-09-10T18:27:51.006434+0000 mon.a (mon.0) 317 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:27:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[18597]: cluster 2024-09-10T18:27:51.006577+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:27:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[18597]: audit 2024-09-10T18:27:51.006825+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[18597]: audit 2024-09-10T18:27:51.007591+0000 mon.a (mon.0) 320 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:27:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[18597]: cluster 2024-09-10T18:27:51.181134+0000 mgr.y (mgr.14150) 143 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[31607]: audit 2024-09-10T18:27:51.006434+0000 mon.a (mon.0) 317 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:27:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[31607]: cluster 2024-09-10T18:27:51.006577+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:27:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[31607]: audit 2024-09-10T18:27:51.006825+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[31607]: audit 2024-09-10T18:27:51.007591+0000 mon.a (mon.0) 320 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:27:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:52 smithi100 bash[31607]: cluster 2024-09-10T18:27:51.181134+0000 mgr.y (mgr.14150) 143 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:52.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:52 smithi151 bash[22565]: audit 2024-09-10T18:27:51.006434+0000 mon.a (mon.0) 317 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:27:52.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:52 smithi151 bash[22565]: cluster 2024-09-10T18:27:51.006577+0000 mon.a (mon.0) 318 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:27:52.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:52 smithi151 bash[22565]: audit 2024-09-10T18:27:51.006825+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:52.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:52 smithi151 bash[22565]: audit 2024-09-10T18:27:51.007591+0000 mon.a (mon.0) 320 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:27:52.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:52 smithi151 bash[22565]: cluster 2024-09-10T18:27:51.181134+0000 mgr.y (mgr.14150) 143 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:27:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[18597]: audit 2024-09-10T18:27:52.019898+0000 mon.a (mon.0) 321 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:27:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[18597]: cluster 2024-09-10T18:27:52.020004+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:27:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[18597]: audit 2024-09-10T18:27:52.020214+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[18597]: audit 2024-09-10T18:27:52.026434+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[31607]: audit 2024-09-10T18:27:52.019898+0000 mon.a (mon.0) 321 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:27:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[31607]: cluster 2024-09-10T18:27:52.020004+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:27:53.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[31607]: audit 2024-09-10T18:27:52.020214+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:53.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:53 smithi100 bash[31607]: audit 2024-09-10T18:27:52.026434+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:53.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:53 smithi151 bash[22565]: audit 2024-09-10T18:27:52.019898+0000 mon.a (mon.0) 321 : audit [INF] from='osd.1 v2:172.21.15.100:6805/59055423' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:27:53.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:53 smithi151 bash[22565]: cluster 2024-09-10T18:27:52.020004+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:27:53.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:53 smithi151 bash[22565]: audit 2024-09-10T18:27:52.020214+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:53.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:53 smithi151 bash[22565]: audit 2024-09-10T18:27:52.026434+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:53.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.513+0000 7f8d02ffd640 1 -- 172.21.15.100:0/261661554 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8d10104060 con 0x7f8cfc06ed40 2024-09-10T18:27:53.520 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8cfc06ed40 msgr2=0x7f8cfc0711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 --2- 172.21.15.100:0/261661554 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8cfc06ed40 0x7f8cfc0711f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c004920 tx=0x7f8d0c009da0 comp rx=0 tx=0).stop 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d100ff9c0 msgr2=0x7f8d100fb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d100ff9c0 0x7f8d100fb380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ebc0 tx=0x7f8d0400b040 comp rx=0 tx=0).stop 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 shutdown_connections 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 --2- 172.21.15.100:0/261661554 >> v2:172.21.15.100:6800/3986579393 conn(0x7f8cfc06ed40 0x7f8cfc0711f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d101024d0 0x7f8d100fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:53.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d100ff9c0 0x7f8d100fb380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 --2- 172.21.15.100:0/261661554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10066c70 0x7f8d100fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:27:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 >> 172.21.15.100:0/261661554 conn(0x7f8d100f6f30 msgr2=0x7f8d10102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:27:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 shutdown_connections 2024-09-10T18:27:53.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:27:53.517+0000 7f8d186c5640 1 -- 172.21.15.100:0/261661554 wait complete. 2024-09-10T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: cluster 2024-09-10T18:27:51.521183+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: cluster 2024-09-10T18:27:51.521275+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: cluster 2024-09-10T18:27:53.022000+0000 mon.a (mon.0) 325 : cluster [INF] osd.1 v2:172.21.15.100:6805/59055423 boot 2024-09-10T18:27:54.041 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: cluster 2024-09-10T18:27:53.022114+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: audit 2024-09-10T18:27:53.023110+0000 mon.a (mon.0) 327 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: cluster 2024-09-10T18:27:53.181712+0000 mgr.y (mgr.14150) 144 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: audit 2024-09-10T18:27:53.499487+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: audit 2024-09-10T18:27:53.508238+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[31607]: audit 2024-09-10T18:27:53.515702+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: cluster 2024-09-10T18:27:51.521183+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: cluster 2024-09-10T18:27:51.521275+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: cluster 2024-09-10T18:27:53.022000+0000 mon.a (mon.0) 325 : cluster [INF] osd.1 v2:172.21.15.100:6805/59055423 boot 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: cluster 2024-09-10T18:27:53.022114+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:27:54.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: audit 2024-09-10T18:27:53.023110+0000 mon.a (mon.0) 327 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:54.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: cluster 2024-09-10T18:27:53.181712+0000 mgr.y (mgr.14150) 144 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:54.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: audit 2024-09-10T18:27:53.499487+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:54.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: audit 2024-09-10T18:27:53.508238+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:54.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:54 smithi100 bash[18597]: audit 2024-09-10T18:27:53.515702+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:54.262 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:27:54.265 INFO:tasks.cephadm:Deploying osd.2 on smithi100 with /dev/vg_nvme/lv_2... 2024-09-10T18:27:54.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:27:54.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: cluster 2024-09-10T18:27:51.521183+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:27:54.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: cluster 2024-09-10T18:27:51.521275+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:27:54.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: cluster 2024-09-10T18:27:53.022000+0000 mon.a (mon.0) 325 : cluster [INF] osd.1 v2:172.21.15.100:6805/59055423 boot 2024-09-10T18:27:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: cluster 2024-09-10T18:27:53.022114+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:27:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: audit 2024-09-10T18:27:53.023110+0000 mon.a (mon.0) 327 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:27:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: cluster 2024-09-10T18:27:53.181712+0000 mgr.y (mgr.14150) 144 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: audit 2024-09-10T18:27:53.499487+0000 mon.a (mon.0) 328 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:27:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: audit 2024-09-10T18:27:53.508238+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:54 smithi151 bash[22565]: audit 2024-09-10T18:27:53.515702+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:27:54.446 INFO:journalctl@ceph.osd.1.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:27:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:55 smithi100 bash[18597]: cluster 2024-09-10T18:27:54.042403+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:27:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:55 smithi100 bash[31607]: cluster 2024-09-10T18:27:54.042403+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:27:55.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:55 smithi151 bash[22565]: cluster 2024-09-10T18:27:54.042403+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:27:56.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:56 smithi151 bash[22565]: cluster 2024-09-10T18:27:55.182191+0000 mgr.y (mgr.14150) 145 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:56.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:56 smithi100 bash[18597]: cluster 2024-09-10T18:27:55.182191+0000 mgr.y (mgr.14150) 145 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:56 smithi100 bash[31607]: cluster 2024-09-10T18:27:55.182191+0000 mgr.y (mgr.14150) 145 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:58.283 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:27:58.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:27:58 smithi100 bash[18597]: cluster 2024-09-10T18:27:57.182701+0000 mgr.y (mgr.14150) 146 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:58.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:27:58 smithi100 bash[31607]: cluster 2024-09-10T18:27:57.182701+0000 mgr.y (mgr.14150) 146 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:27:58.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:27:58 smithi151 bash[22565]: cluster 2024-09-10T18:27:57.182701+0000 mgr.y (mgr.14150) 146 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:00.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:00 smithi100 bash[18597]: cluster 2024-09-10T18:27:59.183189+0000 mgr.y (mgr.14150) 147 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:00 smithi100 bash[31607]: cluster 2024-09-10T18:27:59.183189+0000 mgr.y (mgr.14150) 147 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:00.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:00 smithi151 bash[22565]: cluster 2024-09-10T18:27:59.183189+0000 mgr.y (mgr.14150) 147 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:02.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:02 smithi100 bash[18597]: cluster 2024-09-10T18:28:01.183715+0000 mgr.y (mgr.14150) 148 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:02 smithi100 bash[31607]: cluster 2024-09-10T18:28:01.183715+0000 mgr.y (mgr.14150) 148 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:02.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:02 smithi151 bash[22565]: cluster 2024-09-10T18:28:01.183715+0000 mgr.y (mgr.14150) 148 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:02.993 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:28:03.178 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:28:03.200 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-10T18:28:03.200 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-10T18:28:03.201 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0191051 s, 549 MB/s 2024-09-10T18:28:03.201 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-10T18:28:03.904 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-09-10T18:28:04.251 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:04 smithi100 bash[31607]: cluster 2024-09-10T18:28:03.184251+0000 mgr.y (mgr.14150) 149 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:04.527 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:04 smithi100 bash[18597]: cluster 2024-09-10T18:28:03.184251+0000 mgr.y (mgr.14150) 149 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:04.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:04 smithi151 bash[22565]: cluster 2024-09-10T18:28:03.184251+0000 mgr.y (mgr.14150) 149 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: cephadm 2024-09-10T18:28:05.034645+0000 mgr.y (mgr.14150) 150 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:28:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.043564+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.052502+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.054136+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:28:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.055788+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:28:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: cephadm 2024-09-10T18:28:05.056891+0000 mgr.y (mgr.14150) 151 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 8123M 2024-09-10T18:28:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.064188+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.065854+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.067314+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: audit 2024-09-10T18:28:05.076335+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[18597]: cluster 2024-09-10T18:28:05.184714+0000 mgr.y (mgr.14150) 152 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: cephadm 2024-09-10T18:28:05.034645+0000 mgr.y (mgr.14150) 150 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.043564+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.052502+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.054136+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.055788+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: cephadm 2024-09-10T18:28:05.056891+0000 mgr.y (mgr.14150) 151 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 8123M 2024-09-10T18:28:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.064188+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.065854+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.067314+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: audit 2024-09-10T18:28:05.076335+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:06 smithi100 bash[31607]: cluster 2024-09-10T18:28:05.184714+0000 mgr.y (mgr.14150) 152 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:06.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: cephadm 2024-09-10T18:28:05.034645+0000 mgr.y (mgr.14150) 150 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:28:06.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.043564+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.418 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.052502+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.054136+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.055788+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: cephadm 2024-09-10T18:28:05.056891+0000 mgr.y (mgr.14150) 151 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 8123M 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.064188+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.065854+0000 mon.a (mon.0) 337 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.067314+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: audit 2024-09-10T18:28:05.076335+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:06.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:06 smithi151 bash[22565]: cluster 2024-09-10T18:28:05.184714+0000 mgr.y (mgr.14150) 152 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:08.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:08 smithi100 bash[18597]: cluster 2024-09-10T18:28:07.185222+0000 mgr.y (mgr.14150) 153 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:08.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:08 smithi100 bash[31607]: cluster 2024-09-10T18:28:07.185222+0000 mgr.y (mgr.14150) 153 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:08.641 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:28:08.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:08 smithi151 bash[22565]: cluster 2024-09-10T18:28:07.185222+0000 mgr.y (mgr.14150) 153 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:10 smithi100 bash[18597]: cluster 2024-09-10T18:28:09.185731+0000 mgr.y (mgr.14150) 154 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:10.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:10 smithi100 bash[31607]: cluster 2024-09-10T18:28:09.185731+0000 mgr.y (mgr.14150) 154 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:10.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:10 smithi151 bash[22565]: cluster 2024-09-10T18:28:09.185731+0000 mgr.y (mgr.14150) 154 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:11.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- 172.21.15.100:0/480366049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 msgr2=0x7fd4000fbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:28:11.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 --2- 172.21.15.100:0/480366049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd4000fbb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8004890 tx=0x7fd3f8009da0 comp rx=0 tx=0).stop 2024-09-10T18:28:11.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- 172.21.15.100:0/480366049 shutdown_connections 2024-09-10T18:28:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 --2- 172.21.15.100:0/480366049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4000fd380 0x7fd400103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 --2- 172.21.15.100:0/480366049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4000fc9d0 0x7fd4000fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 --2- 172.21.15.100:0/480366049 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd4000fbb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- 172.21.15.100:0/480366049 >> 172.21.15.100:0/480366049 conn(0x7fd4000f6ea0 msgr2=0x7fd4000f92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:28:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- 172.21.15.100:0/480366049 shutdown_connections 2024-09-10T18:28:11.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- 172.21.15.100:0/480366049 wait complete. 2024-09-10T18:28:11.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 Processor -- start 2024-09-10T18:28:11.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- start start 2024-09-10T18:28:11.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd40010ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:28:11.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4000fc9d0 0x7fd40010f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:28:11.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4000fd380 0x7fd400116500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:28:11.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400116ad0 con 0x7fd4000fd380 2024-09-10T18:28:11.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400116c40 con 0x7fd4000fc9d0 2024-09-10T18:28:11.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd400116e00 con 0x7fd4000fb700 2024-09-10T18:28:11.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd40010ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:28:11.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd40010ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:33214/0 (socket says 172.21.15.100:33214) 2024-09-10T18:28:11.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 -- 172.21.15.100:0/3682911862 learned_addr learned my addr 172.21.15.100:0/3682911862 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:28:11.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fed9d640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4000fd380 0x7fd400116500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:28:11.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fdd9b640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4000fc9d0 0x7fd40010f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:28:11.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 -- 172.21.15.100:0/3682911862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4000fc9d0 msgr2=0x7fd40010f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:28:11.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4000fc9d0 0x7fd40010f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:11.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 -- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4000fd380 msgr2=0x7fd400116500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:28:11.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4000fd380 0x7fd400116500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:11.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 -- 172.21.15.100:0/3682911862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f4005040 con 0x7fd4000fb700 2024-09-10T18:28:11.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fdd9b640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4000fc9d0 0x7fd40010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:28:11.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3fe59c640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd40010ef60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8004860 tx=0x7fd3f8009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:28:11.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3eeffd640 1 -- 172.21.15.100:0/3682911862 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80090d0 con 0x7fd4000fb700 2024-09-10T18:28:11.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3eeffd640 1 -- 172.21.15.100:0/3682911862 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd3f803c8b0 con 0x7fd4000fb700 2024-09-10T18:28:11.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd3eeffd640 1 -- 172.21.15.100:0/3682911862 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8033630 con 0x7fd4000fb700 2024-09-10T18:28:11.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f8005040 con 0x7fd4000fb700 2024-09-10T18:28:11.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.861+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd400117440 con 0x7fd4000fb700 2024-09-10T18:28:11.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.865+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd400106d80 con 0x7fd4000fb700 2024-09-10T18:28:11.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.865+0000 7fd3eeffd640 1 -- 172.21.15.100:0/3682911862 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 14) v1 ==== 90690+0+0 (secure 0 0 0) 0x7fd3f8030020 con 0x7fd4000fb700 2024-09-10T18:28:11.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.865+0000 7fd3eeffd640 1 --2- 172.21.15.100:0/3682911862 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd3e806ed70 0x7fd3e8071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:28:11.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.865+0000 7fd3eeffd640 1 -- 172.21.15.100:0/3682911862 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 1829+0+0 (secure 0 0 0) 0x7fd3f80ad5a0 con 0x7fd4000fb700 2024-09-10T18:28:11.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.865+0000 7fd3fdd9b640 1 --2- 172.21.15.100:0/3682911862 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd3e806ed70 0x7fd3e8071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:28:11.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.865+0000 7fd3fdd9b640 1 --2- 172.21.15.100:0/3682911862 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd3e806ed70 0x7fd3e8071220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd400110460 tx=0x7fd3f4009da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:28:11.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:11.873+0000 7fd3eeffd640 1 -- 172.21.15.100:0/3682911862 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd3f807b730 con 0x7fd4000fb700 2024-09-10T18:28:12.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:12.057+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4001065f0 con 0x7fd3e806ed70 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[18597]: cluster 2024-09-10T18:28:11.186218+0000 mgr.y (mgr.14150) 155 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[18597]: audit 2024-09-10T18:28:12.062591+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[18597]: audit 2024-09-10T18:28:12.067497+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[18597]: audit 2024-09-10T18:28:12.068787+0000 mon.a (mon.0) 342 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[31607]: cluster 2024-09-10T18:28:11.186218+0000 mgr.y (mgr.14150) 155 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[31607]: audit 2024-09-10T18:28:12.062591+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[31607]: audit 2024-09-10T18:28:12.067497+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:28:12.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:12 smithi100 bash[31607]: audit 2024-09-10T18:28:12.068787+0000 mon.a (mon.0) 342 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:12.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:12 smithi151 bash[22565]: cluster 2024-09-10T18:28:11.186218+0000 mgr.y (mgr.14150) 155 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:12.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:12 smithi151 bash[22565]: audit 2024-09-10T18:28:12.062591+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:28:12.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:12 smithi151 bash[22565]: audit 2024-09-10T18:28:12.067497+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:28:12.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:12 smithi151 bash[22565]: audit 2024-09-10T18:28:12.068787+0000 mon.a (mon.0) 342 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:13.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:13 smithi100 bash[18597]: audit 2024-09-10T18:28:12.059914+0000 mgr.y (mgr.14150) 156 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:13.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:13 smithi100 bash[31607]: audit 2024-09-10T18:28:12.059914+0000 mgr.y (mgr.14150) 156 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:13.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:13 smithi151 bash[22565]: audit 2024-09-10T18:28:12.059914+0000 mgr.y (mgr.14150) 156 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:28:14.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:14 smithi100 bash[18597]: cluster 2024-09-10T18:28:13.186703+0000 mgr.y (mgr.14150) 157 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:14.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:14 smithi100 bash[31607]: cluster 2024-09-10T18:28:13.186703+0000 mgr.y (mgr.14150) 157 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:14.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:14 smithi151 bash[22565]: cluster 2024-09-10T18:28:13.186703+0000 mgr.y (mgr.14150) 157 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:16.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:16 smithi100 bash[18597]: cluster 2024-09-10T18:28:15.187176+0000 mgr.y (mgr.14150) 158 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:16 smithi100 bash[31607]: cluster 2024-09-10T18:28:15.187176+0000 mgr.y (mgr.14150) 158 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:16.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:16 smithi151 bash[22565]: cluster 2024-09-10T18:28:15.187176+0000 mgr.y (mgr.14150) 158 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:18.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:18 smithi100 bash[18597]: cluster 2024-09-10T18:28:17.187627+0000 mgr.y (mgr.14150) 159 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:18.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:18 smithi100 bash[31607]: cluster 2024-09-10T18:28:17.187627+0000 mgr.y (mgr.14150) 159 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:18.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:18 smithi151 bash[22565]: cluster 2024-09-10T18:28:17.187627+0000 mgr.y (mgr.14150) 159 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:20.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:20 smithi100 bash[18597]: cluster 2024-09-10T18:28:19.188156+0000 mgr.y (mgr.14150) 160 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:20 smithi100 bash[31607]: cluster 2024-09-10T18:28:19.188156+0000 mgr.y (mgr.14150) 160 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:20.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:20 smithi151 bash[22565]: cluster 2024-09-10T18:28:19.188156+0000 mgr.y (mgr.14150) 160 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:21.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[31607]: audit 2024-09-10T18:28:20.750973+0000 mon.a (mon.0) 343 : audit [INF] from='client.? 172.21.15.100:0/2557986507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce92944f-0b92-46c8-a6b7-6cdd00f9736a"}]: dispatch 2024-09-10T18:28:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[31607]: audit 2024-09-10T18:28:20.760654+0000 mon.a (mon.0) 344 : audit [INF] from='client.? 172.21.15.100:0/2557986507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce92944f-0b92-46c8-a6b7-6cdd00f9736a"}]': finished 2024-09-10T18:28:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[31607]: cluster 2024-09-10T18:28:20.760695+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:28:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[31607]: audit 2024-09-10T18:28:20.760817+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[18597]: audit 2024-09-10T18:28:20.750973+0000 mon.a (mon.0) 343 : audit [INF] from='client.? 172.21.15.100:0/2557986507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce92944f-0b92-46c8-a6b7-6cdd00f9736a"}]: dispatch 2024-09-10T18:28:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[18597]: audit 2024-09-10T18:28:20.760654+0000 mon.a (mon.0) 344 : audit [INF] from='client.? 172.21.15.100:0/2557986507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce92944f-0b92-46c8-a6b7-6cdd00f9736a"}]': finished 2024-09-10T18:28:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[18597]: cluster 2024-09-10T18:28:20.760695+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:28:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:21 smithi100 bash[18597]: audit 2024-09-10T18:28:20.760817+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:21.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:21 smithi151 bash[22565]: audit 2024-09-10T18:28:20.750973+0000 mon.a (mon.0) 343 : audit [INF] from='client.? 172.21.15.100:0/2557986507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce92944f-0b92-46c8-a6b7-6cdd00f9736a"}]: dispatch 2024-09-10T18:28:21.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:21 smithi151 bash[22565]: audit 2024-09-10T18:28:20.760654+0000 mon.a (mon.0) 344 : audit [INF] from='client.? 172.21.15.100:0/2557986507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce92944f-0b92-46c8-a6b7-6cdd00f9736a"}]': finished 2024-09-10T18:28:21.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:21 smithi151 bash[22565]: cluster 2024-09-10T18:28:20.760695+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:28:21.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:21 smithi151 bash[22565]: audit 2024-09-10T18:28:20.760817+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:22 smithi100 bash[18597]: cluster 2024-09-10T18:28:21.188686+0000 mgr.y (mgr.14150) 161 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:22.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:22 smithi100 bash[18597]: audit 2024-09-10T18:28:21.452524+0000 mon.a (mon.0) 347 : audit [DBG] from='client.? 172.21.15.100:0/1447459294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:28:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:22 smithi100 bash[31607]: cluster 2024-09-10T18:28:21.188686+0000 mgr.y (mgr.14150) 161 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:22 smithi100 bash[31607]: audit 2024-09-10T18:28:21.452524+0000 mon.a (mon.0) 347 : audit [DBG] from='client.? 172.21.15.100:0/1447459294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:28:22.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:22 smithi151 bash[22565]: cluster 2024-09-10T18:28:21.188686+0000 mgr.y (mgr.14150) 161 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:22.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:22 smithi151 bash[22565]: audit 2024-09-10T18:28:21.452524+0000 mon.a (mon.0) 347 : audit [DBG] from='client.? 172.21.15.100:0/1447459294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:28:24.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:24 smithi100 bash[18597]: cluster 2024-09-10T18:28:23.189272+0000 mgr.y (mgr.14150) 162 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:24 smithi100 bash[31607]: cluster 2024-09-10T18:28:23.189272+0000 mgr.y (mgr.14150) 162 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:24.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:24 smithi151 bash[22565]: cluster 2024-09-10T18:28:23.189272+0000 mgr.y (mgr.14150) 162 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:26.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:26 smithi100 bash[18597]: cluster 2024-09-10T18:28:25.189856+0000 mgr.y (mgr.14150) 163 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:26.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:26 smithi100 bash[31607]: cluster 2024-09-10T18:28:25.189856+0000 mgr.y (mgr.14150) 163 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:26.668 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:26 smithi151 bash[22565]: cluster 2024-09-10T18:28:25.189856+0000 mgr.y (mgr.14150) 163 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:28.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:28 smithi100 bash[18597]: cluster 2024-09-10T18:28:27.190324+0000 mgr.y (mgr.14150) 164 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:28.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:28 smithi100 bash[31607]: cluster 2024-09-10T18:28:27.190324+0000 mgr.y (mgr.14150) 164 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:28.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:28 smithi151 bash[22565]: cluster 2024-09-10T18:28:27.190324+0000 mgr.y (mgr.14150) 164 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:30.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:30 smithi100 bash[18597]: cluster 2024-09-10T18:28:29.190851+0000 mgr.y (mgr.14150) 165 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:30.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:30 smithi100 bash[31607]: cluster 2024-09-10T18:28:29.190851+0000 mgr.y (mgr.14150) 165 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:30.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:30 smithi151 bash[22565]: cluster 2024-09-10T18:28:29.190851+0000 mgr.y (mgr.14150) 165 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:32.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:32 smithi151 bash[22565]: cluster 2024-09-10T18:28:31.191398+0000 mgr.y (mgr.14150) 166 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:32 smithi100 bash[18597]: cluster 2024-09-10T18:28:31.191398+0000 mgr.y (mgr.14150) 166 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:32 smithi100 bash[31607]: cluster 2024-09-10T18:28:31.191398+0000 mgr.y (mgr.14150) 166 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:34.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:34 smithi151 bash[22565]: cluster 2024-09-10T18:28:33.191882+0000 mgr.y (mgr.14150) 167 : cluster [DBG] pgmap v132: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:34.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:34 smithi151 bash[22565]: audit 2024-09-10T18:28:33.480847+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:28:34.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:34 smithi151 bash[22565]: audit 2024-09-10T18:28:33.482218+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:34.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:34 smithi151 bash[22565]: cephadm 2024-09-10T18:28:33.483444+0000 mgr.y (mgr.14150) 168 : cephadm [INF] Deploying daemon osd.2 on smithi100 2024-09-10T18:28:34.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[18597]: cluster 2024-09-10T18:28:33.191882+0000 mgr.y (mgr.14150) 167 : cluster [DBG] pgmap v132: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:34.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[18597]: audit 2024-09-10T18:28:33.480847+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:28:34.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[18597]: audit 2024-09-10T18:28:33.482218+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:34.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[18597]: cephadm 2024-09-10T18:28:33.483444+0000 mgr.y (mgr.14150) 168 : cephadm [INF] Deploying daemon osd.2 on smithi100 2024-09-10T18:28:34.786 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[31607]: cluster 2024-09-10T18:28:33.191882+0000 mgr.y (mgr.14150) 167 : cluster [DBG] pgmap v132: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:34.786 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[31607]: audit 2024-09-10T18:28:33.480847+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:28:34.786 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[31607]: audit 2024-09-10T18:28:33.482218+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:34.786 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:34 smithi100 bash[31607]: cephadm 2024-09-10T18:28:33.483444+0000 mgr.y (mgr.14150) 168 : cephadm [INF] Deploying daemon osd.2 on smithi100 2024-09-10T18:28:36.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:36 smithi151 bash[22565]: cluster 2024-09-10T18:28:35.192402+0000 mgr.y (mgr.14150) 169 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:36.720 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:36 smithi100 bash[18597]: cluster 2024-09-10T18:28:35.192402+0000 mgr.y (mgr.14150) 169 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:36.720 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:36 smithi100 bash[31607]: cluster 2024-09-10T18:28:35.192402+0000 mgr.y (mgr.14150) 169 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[18597]: cluster 2024-09-10T18:28:37.192886+0000 mgr.y (mgr.14150) 170 : cluster [DBG] pgmap v134: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[18597]: audit 2024-09-10T18:28:38.183584+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[18597]: audit 2024-09-10T18:28:38.189689+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[18597]: audit 2024-09-10T18:28:38.195787+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[31607]: cluster 2024-09-10T18:28:37.192886+0000 mgr.y (mgr.14150) 170 : cluster [DBG] pgmap v134: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[31607]: audit 2024-09-10T18:28:38.183584+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[31607]: audit 2024-09-10T18:28:38.189689+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:38 smithi100 bash[31607]: audit 2024-09-10T18:28:38.195787+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:38.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:38 smithi151 bash[22565]: cluster 2024-09-10T18:28:37.192886+0000 mgr.y (mgr.14150) 170 : cluster [DBG] pgmap v134: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:38.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:38 smithi151 bash[22565]: audit 2024-09-10T18:28:38.183584+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:38.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:38 smithi151 bash[22565]: audit 2024-09-10T18:28:38.189689+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:38.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:38 smithi151 bash[22565]: audit 2024-09-10T18:28:38.195787+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:40.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:40 smithi100 bash[18597]: cluster 2024-09-10T18:28:39.193368+0000 mgr.y (mgr.14150) 171 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:40 smithi100 bash[31607]: cluster 2024-09-10T18:28:39.193368+0000 mgr.y (mgr.14150) 171 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:40.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:40 smithi151 bash[22565]: cluster 2024-09-10T18:28:39.193368+0000 mgr.y (mgr.14150) 171 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:42.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:42 smithi151 bash[22565]: cluster 2024-09-10T18:28:41.193873+0000 mgr.y (mgr.14150) 172 : cluster [DBG] pgmap v136: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:42 smithi100 bash[18597]: cluster 2024-09-10T18:28:41.193873+0000 mgr.y (mgr.14150) 172 : cluster [DBG] pgmap v136: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:42 smithi100 bash[31607]: cluster 2024-09-10T18:28:41.193873+0000 mgr.y (mgr.14150) 172 : cluster [DBG] pgmap v136: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:44.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:44 smithi100 bash[18597]: cluster 2024-09-10T18:28:43.194371+0000 mgr.y (mgr.14150) 173 : cluster [DBG] pgmap v137: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:44 smithi100 bash[31607]: cluster 2024-09-10T18:28:43.194371+0000 mgr.y (mgr.14150) 173 : cluster [DBG] pgmap v137: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:44.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:44 smithi151 bash[22565]: cluster 2024-09-10T18:28:43.194371+0000 mgr.y (mgr.14150) 173 : cluster [DBG] pgmap v137: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:45.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[18597]: audit 2024-09-10T18:28:44.538804+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[18597]: audit 2024-09-10T18:28:44.544940+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[18597]: audit 2024-09-10T18:28:44.916281+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:45.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[18597]: audit 2024-09-10T18:28:44.917412+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:45.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[18597]: audit 2024-09-10T18:28:44.924830+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[18597]: audit 2024-09-10T18:28:45.308549+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 v2:172.21.15.100:6809/370401307' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:28:45.948 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[18597]: audit 2024-09-10T18:28:45.309077+0000 mon.a (mon.0) 358 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:28:45.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[31607]: audit 2024-09-10T18:28:44.538804+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[31607]: audit 2024-09-10T18:28:44.544940+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[31607]: audit 2024-09-10T18:28:44.916281+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:45.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[31607]: audit 2024-09-10T18:28:44.917412+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:45.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[31607]: audit 2024-09-10T18:28:44.924830+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[31607]: audit 2024-09-10T18:28:45.308549+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 v2:172.21.15.100:6809/370401307' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:28:45.949 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:45 smithi100 bash[31607]: audit 2024-09-10T18:28:45.309077+0000 mon.a (mon.0) 358 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:28:45.950 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:45 smithi151 bash[22565]: audit 2024-09-10T18:28:44.538804+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.950 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:45 smithi151 bash[22565]: audit 2024-09-10T18:28:44.544940+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.950 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:45 smithi151 bash[22565]: audit 2024-09-10T18:28:44.916281+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:28:45.950 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:45 smithi151 bash[22565]: audit 2024-09-10T18:28:44.917412+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:28:45.950 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:45 smithi151 bash[22565]: audit 2024-09-10T18:28:44.924830+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:45.950 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:45 smithi151 bash[22565]: audit 2024-09-10T18:28:45.308549+0000 mon.c (mon.2) 8 : audit [INF] from='osd.2 v2:172.21.15.100:6809/370401307' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:28:45.950 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:45 smithi151 bash[22565]: audit 2024-09-10T18:28:45.309077+0000 mon.a (mon.0) 358 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:28:46.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[18597]: cluster 2024-09-10T18:28:45.194874+0000 mgr.y (mgr.14150) 174 : cluster [DBG] pgmap v138: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[18597]: audit 2024-09-10T18:28:45.934359+0000 mon.a (mon.0) 359 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[18597]: cluster 2024-09-10T18:28:45.934456+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[18597]: audit 2024-09-10T18:28:45.934674+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[18597]: audit 2024-09-10T18:28:45.935902+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 v2:172.21.15.100:6809/370401307' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[18597]: audit 2024-09-10T18:28:45.936646+0000 mon.a (mon.0) 362 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[31607]: cluster 2024-09-10T18:28:45.194874+0000 mgr.y (mgr.14150) 174 : cluster [DBG] pgmap v138: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[31607]: audit 2024-09-10T18:28:45.934359+0000 mon.a (mon.0) 359 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[31607]: cluster 2024-09-10T18:28:45.934456+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[31607]: audit 2024-09-10T18:28:45.934674+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[31607]: audit 2024-09-10T18:28:45.935902+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 v2:172.21.15.100:6809/370401307' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:28:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:46 smithi100 bash[31607]: audit 2024-09-10T18:28:45.936646+0000 mon.a (mon.0) 362 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:28:46.919 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:46 smithi151 bash[22565]: cluster 2024-09-10T18:28:45.194874+0000 mgr.y (mgr.14150) 174 : cluster [DBG] pgmap v138: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:46.919 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:46 smithi151 bash[22565]: audit 2024-09-10T18:28:45.934359+0000 mon.a (mon.0) 359 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:28:46.919 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:46 smithi151 bash[22565]: cluster 2024-09-10T18:28:45.934456+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:28:46.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:46 smithi151 bash[22565]: audit 2024-09-10T18:28:45.934674+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:46.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:46 smithi151 bash[22565]: audit 2024-09-10T18:28:45.935902+0000 mon.c (mon.2) 9 : audit [INF] from='osd.2 v2:172.21.15.100:6809/370401307' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:28:46.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:46 smithi151 bash[22565]: audit 2024-09-10T18:28:45.936646+0000 mon.a (mon.0) 362 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:28:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[31607]: audit 2024-09-10T18:28:46.937336+0000 mon.a (mon.0) 363 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:28:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[31607]: cluster 2024-09-10T18:28:46.937592+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:28:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[31607]: audit 2024-09-10T18:28:46.937884+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[31607]: audit 2024-09-10T18:28:46.945270+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[31607]: cluster 2024-09-10T18:28:47.195348+0000 mgr.y (mgr.14150) 175 : cluster [DBG] pgmap v141: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[18597]: audit 2024-09-10T18:28:46.937336+0000 mon.a (mon.0) 363 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:28:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[18597]: cluster 2024-09-10T18:28:46.937592+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:28:48.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[18597]: audit 2024-09-10T18:28:46.937884+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:48.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[18597]: audit 2024-09-10T18:28:46.945270+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:48.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:47 smithi100 bash[18597]: cluster 2024-09-10T18:28:47.195348+0000 mgr.y (mgr.14150) 175 : cluster [DBG] pgmap v141: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:48.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:47 smithi151 bash[22565]: audit 2024-09-10T18:28:46.937336+0000 mon.a (mon.0) 363 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:28:48.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:47 smithi151 bash[22565]: cluster 2024-09-10T18:28:46.937592+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:28:48.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:47 smithi151 bash[22565]: audit 2024-09-10T18:28:46.937884+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:48.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:47 smithi151 bash[22565]: audit 2024-09-10T18:28:46.945270+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:48.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:47 smithi151 bash[22565]: cluster 2024-09-10T18:28:47.195348+0000 mgr.y (mgr.14150) 175 : cluster [DBG] pgmap v141: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:28:48.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.596+0000 7fd3eeffd640 1 -- 172.21.15.100:0/3682911862 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd4001065f0 con 0x7fd3e806ed70 2024-09-10T18:28:48.599 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 2 on host 'smithi100' 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd3e806ed70 msgr2=0x7fd3e8071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 --2- 172.21.15.100:0/3682911862 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd3e806ed70 0x7fd3e8071220 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd400110460 tx=0x7fd3f4009da0 comp rx=0 tx=0).stop 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 msgr2=0x7fd40010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd40010ef60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8004860 tx=0x7fd3f8009da0 comp rx=0 tx=0).stop 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 shutdown_connections 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 --2- 172.21.15.100:0/3682911862 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd3e806ed70 0x7fd3e8071220 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4000fd380 0x7fd400116500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4000fc9d0 0x7fd40010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:48.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 --2- 172.21.15.100:0/3682911862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4000fb700 0x7fd40010ef60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:28:48.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 >> 172.21.15.100:0/3682911862 conn(0x7fd4000f6ea0 msgr2=0x7fd4000656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:28:48.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 shutdown_connections 2024-09-10T18:28:48.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:28:48.600+0000 7fd404e2b640 1 -- 172.21.15.100:0/3682911862 wait complete. 2024-09-10T18:28:49.280 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: cluster 2024-09-10T18:28:46.335523+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: cluster 2024-09-10T18:28:46.335617+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: cluster 2024-09-10T18:28:47.940514+0000 mon.a (mon.0) 367 : cluster [INF] osd.2 v2:172.21.15.100:6809/370401307 boot 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: cluster 2024-09-10T18:28:47.940609+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: audit 2024-09-10T18:28:47.941180+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: audit 2024-09-10T18:28:48.580966+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: audit 2024-09-10T18:28:48.588703+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[31607]: audit 2024-09-10T18:28:48.594805+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:49.281 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: cluster 2024-09-10T18:28:46.335523+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:28:49.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: cluster 2024-09-10T18:28:46.335617+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:28:49.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: cluster 2024-09-10T18:28:47.940514+0000 mon.a (mon.0) 367 : cluster [INF] osd.2 v2:172.21.15.100:6809/370401307 boot 2024-09-10T18:28:49.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: cluster 2024-09-10T18:28:47.940609+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:28:49.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: audit 2024-09-10T18:28:47.941180+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:49.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: audit 2024-09-10T18:28:48.580966+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:49.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: audit 2024-09-10T18:28:48.588703+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:49.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:48 smithi100 bash[18597]: audit 2024-09-10T18:28:48.594805+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:49.284 DEBUG:teuthology.orchestra.run.smithi100:osd.2> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:28:49.286 INFO:tasks.cephadm:Deploying osd.3 on smithi100 with /dev/vg_nvme/lv_1... 2024-09-10T18:28:49.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:28:49.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: cluster 2024-09-10T18:28:46.335523+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:28:49.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: cluster 2024-09-10T18:28:46.335617+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:28:49.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: cluster 2024-09-10T18:28:47.940514+0000 mon.a (mon.0) 367 : cluster [INF] osd.2 v2:172.21.15.100:6809/370401307 boot 2024-09-10T18:28:49.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: cluster 2024-09-10T18:28:47.940609+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:28:49.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: audit 2024-09-10T18:28:47.941180+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:28:49.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: audit 2024-09-10T18:28:48.580966+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:28:49.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: audit 2024-09-10T18:28:48.588703+0000 mon.a (mon.0) 371 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:49.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:48 smithi151 bash[22565]: audit 2024-09-10T18:28:48.594805+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:49.460 INFO:journalctl@ceph.osd.2.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:28:50.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:49 smithi100 bash[18597]: cluster 2024-09-10T18:28:48.950500+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:28:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:49 smithi100 bash[18597]: cluster 2024-09-10T18:28:49.195834+0000 mgr.y (mgr.14150) 176 : cluster [DBG] pgmap v144: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:49 smithi100 bash[18597]: audit 2024-09-10T18:28:49.292087+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:28:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:49 smithi100 bash[31607]: cluster 2024-09-10T18:28:48.950500+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:28:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:49 smithi100 bash[31607]: cluster 2024-09-10T18:28:49.195834+0000 mgr.y (mgr.14150) 176 : cluster [DBG] pgmap v144: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:49 smithi100 bash[31607]: audit 2024-09-10T18:28:49.292087+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:28:50.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:49 smithi151 bash[22565]: cluster 2024-09-10T18:28:48.950500+0000 mon.a (mon.0) 373 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:28:50.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:49 smithi151 bash[22565]: cluster 2024-09-10T18:28:49.195834+0000 mgr.y (mgr.14150) 176 : cluster [DBG] pgmap v144: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:50.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:49 smithi151 bash[22565]: audit 2024-09-10T18:28:49.292087+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:28:51.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:50 smithi100 bash[18597]: audit 2024-09-10T18:28:49.969445+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:28:51.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:50 smithi100 bash[18597]: cluster 2024-09-10T18:28:49.969640+0000 mon.a (mon.0) 376 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:28:51.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:50 smithi100 bash[18597]: audit 2024-09-10T18:28:49.971096+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:28:51.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:50 smithi100 bash[31607]: audit 2024-09-10T18:28:49.969445+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:28:51.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:50 smithi100 bash[31607]: cluster 2024-09-10T18:28:49.969640+0000 mon.a (mon.0) 376 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:28:51.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:50 smithi100 bash[31607]: audit 2024-09-10T18:28:49.971096+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:28:51.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:50 smithi151 bash[22565]: audit 2024-09-10T18:28:49.969445+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:28:51.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:50 smithi151 bash[22565]: cluster 2024-09-10T18:28:49.969640+0000 mon.a (mon.0) 376 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:28:51.420 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:50 smithi151 bash[22565]: audit 2024-09-10T18:28:49.971096+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:28:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:51 smithi100 bash[31607]: audit 2024-09-10T18:28:50.970119+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:28:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:51 smithi100 bash[31607]: cluster 2024-09-10T18:28:50.970252+0000 mon.a (mon.0) 379 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:28:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:51 smithi100 bash[31607]: cluster 2024-09-10T18:28:51.196349+0000 mgr.y (mgr.14150) 177 : cluster [DBG] pgmap v147: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:51 smithi100 bash[18597]: audit 2024-09-10T18:28:50.970119+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:28:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:51 smithi100 bash[18597]: cluster 2024-09-10T18:28:50.970252+0000 mon.a (mon.0) 379 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:28:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:51 smithi100 bash[18597]: cluster 2024-09-10T18:28:51.196349+0000 mgr.y (mgr.14150) 177 : cluster [DBG] pgmap v147: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:52.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:51 smithi151 bash[22565]: audit 2024-09-10T18:28:50.970119+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:28:52.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:51 smithi151 bash[22565]: cluster 2024-09-10T18:28:50.970252+0000 mon.a (mon.0) 379 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:28:52.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:51 smithi151 bash[22565]: cluster 2024-09-10T18:28:51.196349+0000 mgr.y (mgr.14150) 177 : cluster [DBG] pgmap v147: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:52 smithi100 bash[18597]: cluster 2024-09-10T18:28:51.986555+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2024-09-10T18:28:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:52 smithi100 bash[18597]: audit 2024-09-10T18:28:52.739522+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:52 smithi100 bash[31607]: cluster 2024-09-10T18:28:51.986555+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2024-09-10T18:28:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:52 smithi100 bash[31607]: audit 2024-09-10T18:28:52.739522+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:28:53.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:52 smithi151 bash[22565]: cluster 2024-09-10T18:28:51.986555+0000 mon.a (mon.0) 380 : cluster [DBG] osdmap e22: 3 total, 3 up, 3 in 2024-09-10T18:28:53.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:52 smithi151 bash[22565]: audit 2024-09-10T18:28:52.739522+0000 mon.a (mon.0) 381 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:54 smithi100 bash[18597]: cluster 2024-09-10T18:28:53.004196+0000 mon.a (mon.0) 382 : cluster [DBG] mgrmap e15: y(active, since 4m), standbys: x 2024-09-10T18:28:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:54 smithi100 bash[18597]: cluster 2024-09-10T18:28:53.196836+0000 mgr.y (mgr.14150) 178 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:54 smithi100 bash[31607]: cluster 2024-09-10T18:28:53.004196+0000 mon.a (mon.0) 382 : cluster [DBG] mgrmap e15: y(active, since 4m), standbys: x 2024-09-10T18:28:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:54 smithi100 bash[31607]: cluster 2024-09-10T18:28:53.196836+0000 mgr.y (mgr.14150) 178 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:53 smithi151 bash[22565]: cluster 2024-09-10T18:28:53.004196+0000 mon.a (mon.0) 382 : cluster [DBG] mgrmap e15: y(active, since 4m), standbys: x 2024-09-10T18:28:54.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:53 smithi151 bash[22565]: cluster 2024-09-10T18:28:53.196836+0000 mgr.y (mgr.14150) 178 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:56.566 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:54.543602+0000 mon.a (mon.0) 383 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:56.566 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:55.416416+0000 mon.a (mon.0) 384 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:56.566 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:55.416746+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:56.566 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:55.417399+0000 mon.a (mon.0) 386 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:56.567 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:55.417695+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:56.567 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:55.420678+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:56.567 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:55.420859+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:56.567 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[31607]: audit 2024-09-10T18:28:55.420987+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:56.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:54.543602+0000 mon.a (mon.0) 383 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:56.567 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:55.416416+0000 mon.a (mon.0) 384 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:56.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:55.416746+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:56.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:55.417399+0000 mon.a (mon.0) 386 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:56.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:55.417695+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:56.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:55.420678+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:56.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:55.420859+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:56.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:56 smithi100 bash[18597]: audit 2024-09-10T18:28:55.420987+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:56.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:54.543602+0000 mon.a (mon.0) 383 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:56.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:55.416416+0000 mon.a (mon.0) 384 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:56.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:55.416746+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:56.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:55.417399+0000 mon.a (mon.0) 386 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:56.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:55.417695+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:56.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:55.420678+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:56.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:55.420859+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:56.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:56 smithi151 bash[22565]: audit 2024-09-10T18:28:55.420987+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: cluster 2024-09-10T18:28:55.197134+0000 mgr.y (mgr.14150) 179 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:55.413159+0000 mon.b (mon.1) 8 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:56.305648+0000 mon.b (mon.1) 9 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:56.343747+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:56.345412+0000 mon.c (mon.2) 10 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:56.345699+0000 mon.a (mon.0) 392 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:56.345917+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:56.346050+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:57 smithi151 bash[22565]: audit 2024-09-10T18:28:57.224315+0000 mon.c (mon.2) 11 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:57.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: cluster 2024-09-10T18:28:55.197134+0000 mgr.y (mgr.14150) 179 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:55.413159+0000 mon.b (mon.1) 8 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:56.305648+0000 mon.b (mon.1) 9 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:56.343747+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:56.345412+0000 mon.c (mon.2) 10 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:56.345699+0000 mon.a (mon.0) 392 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:56.345917+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:56.346050+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[31607]: audit 2024-09-10T18:28:57.224315+0000 mon.c (mon.2) 11 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: cluster 2024-09-10T18:28:55.197134+0000 mgr.y (mgr.14150) 179 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:55.413159+0000 mon.b (mon.1) 8 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:56.305648+0000 mon.b (mon.1) 9 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:57.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:56.343747+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:28:57.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:56.345412+0000 mon.c (mon.2) 10 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:28:57.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:56.345699+0000 mon.a (mon.0) 392 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:28:57.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:56.345917+0000 mon.a (mon.0) 393 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:28:57.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:56.346050+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:28:57.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:57 smithi100 bash[18597]: audit 2024-09-10T18:28:57.224315+0000 mon.c (mon.2) 11 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:28:58.282 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:28:58.457 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:28:58.476 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-09-10T18:28:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-09-10T18:28:58.476 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0165476 s, 634 MB/s 2024-09-10T18:28:58.476 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-09-10T18:28:58.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:28:58 smithi151 bash[22565]: cluster 2024-09-10T18:28:57.197663+0000 mgr.y (mgr.14150) 180 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:58.718 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:28:58 smithi100 bash[31607]: cluster 2024-09-10T18:28:57.197663+0000 mgr.y (mgr.14150) 180 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:58.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:28:58 smithi100 bash[18597]: cluster 2024-09-10T18:28:57.197663+0000 mgr.y (mgr.14150) 180 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:28:59.217 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: cluster 2024-09-10T18:28:59.198227+0000 mgr.y (mgr.14150) 181 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.251689+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.258361+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.259949+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.261417+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.262704+0000 mon.a (mon.0) 399 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.270853+0000 mon.a (mon.0) 400 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.272990+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:00.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.274577+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:00 smithi151 bash[22565]: audit 2024-09-10T18:29:00.282718+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: cluster 2024-09-10T18:28:59.198227+0000 mgr.y (mgr.14150) 181 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.251689+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.258361+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.259949+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.261417+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.262704+0000 mon.a (mon.0) 399 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.270853+0000 mon.a (mon.0) 400 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.272990+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.274577+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[18597]: audit 2024-09-10T18:29:00.282718+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: cluster 2024-09-10T18:28:59.198227+0000 mgr.y (mgr.14150) 181 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.251689+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.258361+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.259949+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.261417+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.262704+0000 mon.a (mon.0) 399 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.270853+0000 mon.a (mon.0) 400 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.272990+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.274577+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:00 smithi100 bash[31607]: audit 2024-09-10T18:29:00.282718+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:01.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:01 smithi151 bash[22565]: cephadm 2024-09-10T18:29:00.243217+0000 mgr.y (mgr.14150) 182 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:29:01.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:01 smithi151 bash[22565]: cephadm 2024-09-10T18:29:00.263723+0000 mgr.y (mgr.14150) 183 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 5415M 2024-09-10T18:29:01.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:01 smithi100 bash[18597]: cephadm 2024-09-10T18:29:00.243217+0000 mgr.y (mgr.14150) 182 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:29:01.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:01 smithi100 bash[18597]: cephadm 2024-09-10T18:29:00.263723+0000 mgr.y (mgr.14150) 183 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 5415M 2024-09-10T18:29:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:01 smithi100 bash[31607]: cephadm 2024-09-10T18:29:00.243217+0000 mgr.y (mgr.14150) 182 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:29:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:01 smithi100 bash[31607]: cephadm 2024-09-10T18:29:00.263723+0000 mgr.y (mgr.14150) 183 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 5415M 2024-09-10T18:29:02.669 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:02 smithi151 bash[22565]: cluster 2024-09-10T18:29:01.198770+0000 mgr.y (mgr.14150) 184 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-10T18:29:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:02 smithi100 bash[18597]: cluster 2024-09-10T18:29:01.198770+0000 mgr.y (mgr.14150) 184 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-10T18:29:02.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:02 smithi100 bash[31607]: cluster 2024-09-10T18:29:01.198770+0000 mgr.y (mgr.14150) 184 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-10T18:29:03.973 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:29:04.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:04 smithi151 bash[22565]: cluster 2024-09-10T18:29:03.199408+0000 mgr.y (mgr.14150) 185 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:29:04.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:04 smithi100 bash[18597]: cluster 2024-09-10T18:29:03.199408+0000 mgr.y (mgr.14150) 185 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:29:04.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:04 smithi100 bash[31607]: cluster 2024-09-10T18:29:03.199408+0000 mgr.y (mgr.14150) 185 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-10T18:29:06.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:06 smithi100 bash[31607]: cluster 2024-09-10T18:29:05.199912+0000 mgr.y (mgr.14150) 186 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:06 smithi100 bash[18597]: cluster 2024-09-10T18:29:05.199912+0000 mgr.y (mgr.14150) 186 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:06.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:06 smithi151 bash[22565]: cluster 2024-09-10T18:29:05.199912+0000 mgr.y (mgr.14150) 186 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- 172.21.15.100:0/1677168171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fde70 msgr2=0x7ff6180fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 --2- 172.21.15.100:0/1677168171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fde70 0x7ff6180fe280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff608009d00 tx=0x7ff608004d10 comp rx=0 tx=0).stop 2024-09-10T18:29:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- 172.21.15.100:0/1677168171 shutdown_connections 2024-09-10T18:29:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 --2- 172.21.15.100:0/1677168171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6180fb050 0x7ff618104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:07.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 --2- 172.21.15.100:0/1677168171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6180fe7c0 0x7ff6180fa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 --2- 172.21.15.100:0/1677168171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fde70 0x7ff6180fe280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- 172.21.15.100:0/1677168171 >> 172.21.15.100:0/1677168171 conn(0x7ff6180f6690 msgr2=0x7ff6180f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:29:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- 172.21.15.100:0/1677168171 shutdown_connections 2024-09-10T18:29:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- 172.21.15.100:0/1677168171 wait complete. 2024-09-10T18:29:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 Processor -- start 2024-09-10T18:29:07.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- start start 2024-09-10T18:29:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6180fb050 0x7ff6180fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6180fde70 0x7ff6180fbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fe7c0 0x7ff6180fc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180fc730 con 0x7ff6180fe7c0 2024-09-10T18:29:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180fc8a0 con 0x7ff6180fb050 2024-09-10T18:29:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff61dd55640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff618116f70 con 0x7ff6180fde70 2024-09-10T18:29:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff617fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fe7c0 0x7ff6180fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff617fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fe7c0 0x7ff6180fc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49202/0 (socket says 172.21.15.100:49202) 2024-09-10T18:29:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff617fff640 1 -- 172.21.15.100:0/3629414130 learned_addr learned my addr 172.21.15.100:0/3629414130 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:29:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff6177fe640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6180fb050 0x7ff6180fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.279+0000 7ff616ffd640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6180fde70 0x7ff6180fbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff617fff640 1 -- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6180fde70 msgr2=0x7ff6180fbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff617fff640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6180fde70 0x7ff6180fbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff617fff640 1 -- 172.21.15.100:0/3629414130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6180fb050 msgr2=0x7ff6180fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff617fff640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6180fb050 0x7ff6180fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff617fff640 1 -- 172.21.15.100:0/3629414130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080098b0 con 0x7ff6180fe7c0 2024-09-10T18:29:07.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff617fff640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fe7c0 0x7ff6180fc090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff6100028f0 tx=0x7ff610004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:29:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff60ffff640 1 -- 172.21.15.100:0/3629414130 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610007c30 con 0x7ff6180fe7c0 2024-09-10T18:29:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff60ffff640 1 -- 172.21.15.100:0/3629414130 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7ff61000f040 con 0x7ff6180fe7c0 2024-09-10T18:29:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff618117170 con 0x7ff6180fe7c0 2024-09-10T18:29:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff60ffff640 1 -- 172.21.15.100:0/3629414130 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000d920 con 0x7ff6180fe7c0 2024-09-10T18:29:07.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff618117640 con 0x7ff6180fe7c0 2024-09-10T18:29:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.283+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff618107ea0 con 0x7ff6180fe7c0 2024-09-10T18:29:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.287+0000 7ff60ffff640 1 -- 172.21.15.100:0/3629414130 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7ff61000dd50 con 0x7ff6180fe7c0 2024-09-10T18:29:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.287+0000 7ff60ffff640 1 --2- 172.21.15.100:0/3629414130 >> v2:172.21.15.100:6800/3986579393 conn(0x7ff60406edd0 0x7ff604071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:29:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.287+0000 7ff6177fe640 1 --2- 172.21.15.100:0/3629414130 >> v2:172.21.15.100:6800/3986579393 conn(0x7ff60406edd0 0x7ff604071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:29:07.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.287+0000 7ff60ffff640 1 -- 172.21.15.100:0/3629414130 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 2507+0+0 (secure 0 0 0) 0x7ff61008d490 con 0x7ff6180fe7c0 2024-09-10T18:29:07.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.287+0000 7ff6177fe640 1 --2- 172.21.15.100:0/3629414130 >> v2:172.21.15.100:6800/3986579393 conn(0x7ff60406edd0 0x7ff604071280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff608009640 tx=0x7ff608035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:29:07.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.291+0000 7ff60ffff640 1 -- 172.21.15.100:0/3629414130 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff61005b1c0 con 0x7ff6180fe7c0 2024-09-10T18:29:07.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:07.483+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6181076b0 con 0x7ff60406edd0 2024-09-10T18:29:08.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:08 smithi151 bash[22565]: cluster 2024-09-10T18:29:07.200425+0000 mgr.y (mgr.14150) 187 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:08.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:08 smithi151 bash[22565]: audit 2024-09-10T18:29:07.488431+0000 mgr.y (mgr.14150) 188 : audit [DBG] from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:08.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:08 smithi151 bash[22565]: audit 2024-09-10T18:29:07.494052+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:29:08.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:08 smithi151 bash[22565]: audit 2024-09-10T18:29:07.497192+0000 mon.a (mon.0) 405 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:29:08.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:08 smithi151 bash[22565]: audit 2024-09-10T18:29:07.498256+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[18597]: cluster 2024-09-10T18:29:07.200425+0000 mgr.y (mgr.14150) 187 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[18597]: audit 2024-09-10T18:29:07.488431+0000 mgr.y (mgr.14150) 188 : audit [DBG] from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[18597]: audit 2024-09-10T18:29:07.494052+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[18597]: audit 2024-09-10T18:29:07.497192+0000 mon.a (mon.0) 405 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[18597]: audit 2024-09-10T18:29:07.498256+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[31607]: cluster 2024-09-10T18:29:07.200425+0000 mgr.y (mgr.14150) 187 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[31607]: audit 2024-09-10T18:29:07.488431+0000 mgr.y (mgr.14150) 188 : audit [DBG] from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[31607]: audit 2024-09-10T18:29:07.494052+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[31607]: audit 2024-09-10T18:29:07.497192+0000 mon.a (mon.0) 405 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:29:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:08 smithi100 bash[31607]: audit 2024-09-10T18:29:07.498256+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:10.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:10 smithi151 bash[22565]: cluster 2024-09-10T18:29:09.200977+0000 mgr.y (mgr.14150) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:10.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:10 smithi100 bash[31607]: cluster 2024-09-10T18:29:09.200977+0000 mgr.y (mgr.14150) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:10 smithi100 bash[18597]: cluster 2024-09-10T18:29:09.200977+0000 mgr.y (mgr.14150) 189 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-10T18:29:12.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:12 smithi151 bash[22565]: cluster 2024-09-10T18:29:11.201537+0000 mgr.y (mgr.14150) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:12 smithi100 bash[18597]: cluster 2024-09-10T18:29:11.201537+0000 mgr.y (mgr.14150) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:12 smithi100 bash[31607]: cluster 2024-09-10T18:29:11.201537+0000 mgr.y (mgr.14150) 190 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:14.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:14 smithi151 bash[22565]: cluster 2024-09-10T18:29:13.202047+0000 mgr.y (mgr.14150) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:14.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:14 smithi100 bash[18597]: cluster 2024-09-10T18:29:13.202047+0000 mgr.y (mgr.14150) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:14.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:14 smithi100 bash[31607]: cluster 2024-09-10T18:29:13.202047+0000 mgr.y (mgr.14150) 191 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:16.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:16 smithi151 bash[22565]: cluster 2024-09-10T18:29:15.202646+0000 mgr.y (mgr.14150) 192 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:16.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:16 smithi151 bash[22565]: audit 2024-09-10T18:29:16.287639+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.100:0/1057462204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]: dispatch 2024-09-10T18:29:16.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:16 smithi151 bash[22565]: audit 2024-09-10T18:29:16.288279+0000 mon.a (mon.0) 407 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]: dispatch 2024-09-10T18:29:16.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:16 smithi151 bash[22565]: audit 2024-09-10T18:29:16.295507+0000 mon.a (mon.0) 408 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]': finished 2024-09-10T18:29:16.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:16 smithi151 bash[22565]: cluster 2024-09-10T18:29:16.295590+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:29:16.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:16 smithi151 bash[22565]: audit 2024-09-10T18:29:16.295797+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[18597]: cluster 2024-09-10T18:29:15.202646+0000 mgr.y (mgr.14150) 192 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[18597]: audit 2024-09-10T18:29:16.287639+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.100:0/1057462204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]: dispatch 2024-09-10T18:29:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[18597]: audit 2024-09-10T18:29:16.288279+0000 mon.a (mon.0) 407 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]: dispatch 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[18597]: audit 2024-09-10T18:29:16.295507+0000 mon.a (mon.0) 408 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]': finished 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[18597]: cluster 2024-09-10T18:29:16.295590+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[18597]: audit 2024-09-10T18:29:16.295797+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[31607]: cluster 2024-09-10T18:29:15.202646+0000 mgr.y (mgr.14150) 192 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[31607]: audit 2024-09-10T18:29:16.287639+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.100:0/1057462204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]: dispatch 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[31607]: audit 2024-09-10T18:29:16.288279+0000 mon.a (mon.0) 407 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]: dispatch 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[31607]: audit 2024-09-10T18:29:16.295507+0000 mon.a (mon.0) 408 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8340f7e5-8d39-4316-ae1e-809bcf286e32"}]': finished 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[31607]: cluster 2024-09-10T18:29:16.295590+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:29:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:16 smithi100 bash[31607]: audit 2024-09-10T18:29:16.295797+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:17.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:17 smithi151 bash[22565]: audit 2024-09-10T18:29:16.955080+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.100:0/3330254753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:29:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:17 smithi100 bash[18597]: audit 2024-09-10T18:29:16.955080+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.100:0/3330254753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:29:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:17 smithi100 bash[31607]: audit 2024-09-10T18:29:16.955080+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.100:0/3330254753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:29:18.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:18 smithi151 bash[22565]: cluster 2024-09-10T18:29:17.203218+0000 mgr.y (mgr.14150) 193 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:18.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:18 smithi100 bash[18597]: cluster 2024-09-10T18:29:17.203218+0000 mgr.y (mgr.14150) 193 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:18 smithi100 bash[31607]: cluster 2024-09-10T18:29:17.203218+0000 mgr.y (mgr.14150) 193 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:20.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:20 smithi151 bash[22565]: cluster 2024-09-10T18:29:19.203680+0000 mgr.y (mgr.14150) 194 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:20.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:20 smithi100 bash[18597]: cluster 2024-09-10T18:29:19.203680+0000 mgr.y (mgr.14150) 194 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:20.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:20 smithi100 bash[31607]: cluster 2024-09-10T18:29:19.203680+0000 mgr.y (mgr.14150) 194 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:22.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:22 smithi151 bash[22565]: cluster 2024-09-10T18:29:21.204197+0000 mgr.y (mgr.14150) 195 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:22 smithi100 bash[18597]: cluster 2024-09-10T18:29:21.204197+0000 mgr.y (mgr.14150) 195 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:22.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:22 smithi100 bash[31607]: cluster 2024-09-10T18:29:21.204197+0000 mgr.y (mgr.14150) 195 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:24.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:24 smithi151 bash[22565]: cluster 2024-09-10T18:29:23.204725+0000 mgr.y (mgr.14150) 196 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:24 smithi100 bash[18597]: cluster 2024-09-10T18:29:23.204725+0000 mgr.y (mgr.14150) 196 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:24.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:24 smithi100 bash[31607]: cluster 2024-09-10T18:29:23.204725+0000 mgr.y (mgr.14150) 196 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:26.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:26 smithi151 bash[22565]: cluster 2024-09-10T18:29:25.205285+0000 mgr.y (mgr.14150) 197 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:26 smithi100 bash[18597]: cluster 2024-09-10T18:29:25.205285+0000 mgr.y (mgr.14150) 197 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:26 smithi100 bash[31607]: cluster 2024-09-10T18:29:25.205285+0000 mgr.y (mgr.14150) 197 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:28.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:28 smithi151 bash[22565]: cluster 2024-09-10T18:29:27.205871+0000 mgr.y (mgr.14150) 198 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:28.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:28 smithi100 bash[18597]: cluster 2024-09-10T18:29:27.205871+0000 mgr.y (mgr.14150) 198 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:28.747 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:28 smithi100 bash[31607]: cluster 2024-09-10T18:29:27.205871+0000 mgr.y (mgr.14150) 198 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:29 smithi100 bash[18597]: audit 2024-09-10T18:29:29.262592+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:29:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:29 smithi100 bash[18597]: audit 2024-09-10T18:29:29.264016+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:29 smithi100 bash[31607]: audit 2024-09-10T18:29:29.262592+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:29:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:29 smithi100 bash[31607]: audit 2024-09-10T18:29:29.264016+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:29.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:29 smithi151 bash[22565]: audit 2024-09-10T18:29:29.262592+0000 mon.a (mon.0) 411 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:29:29.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:29 smithi151 bash[22565]: audit 2024-09-10T18:29:29.264016+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:30.717 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:30 smithi100 bash[18597]: cluster 2024-09-10T18:29:29.206470+0000 mgr.y (mgr.14150) 199 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:30.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:30 smithi100 bash[18597]: cephadm 2024-09-10T18:29:29.265183+0000 mgr.y (mgr.14150) 200 : cephadm [INF] Deploying daemon osd.3 on smithi100 2024-09-10T18:29:30.718 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:30 smithi100 bash[31607]: cluster 2024-09-10T18:29:29.206470+0000 mgr.y (mgr.14150) 199 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:30.718 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:30 smithi100 bash[31607]: cephadm 2024-09-10T18:29:29.265183+0000 mgr.y (mgr.14150) 200 : cephadm [INF] Deploying daemon osd.3 on smithi100 2024-09-10T18:29:30.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:30 smithi151 bash[22565]: cluster 2024-09-10T18:29:29.206470+0000 mgr.y (mgr.14150) 199 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:30.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:30 smithi151 bash[22565]: cephadm 2024-09-10T18:29:29.265183+0000 mgr.y (mgr.14150) 200 : cephadm [INF] Deploying daemon osd.3 on smithi100 2024-09-10T18:29:32.752 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:32 smithi100 bash[18597]: cluster 2024-09-10T18:29:31.207105+0000 mgr.y (mgr.14150) 201 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:32.753 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:32 smithi100 bash[31607]: cluster 2024-09-10T18:29:31.207105+0000 mgr.y (mgr.14150) 201 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:32.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:32 smithi151 bash[22565]: cluster 2024-09-10T18:29:31.207105+0000 mgr.y (mgr.14150) 201 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:34.621 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[18597]: cluster 2024-09-10T18:29:33.207765+0000 mgr.y (mgr.14150) 202 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:34.621 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[18597]: audit 2024-09-10T18:29:34.259505+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:34.621 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[18597]: audit 2024-09-10T18:29:34.264293+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:34.621 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[18597]: audit 2024-09-10T18:29:34.271465+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[31607]: cluster 2024-09-10T18:29:33.207765+0000 mgr.y (mgr.14150) 202 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[31607]: audit 2024-09-10T18:29:34.259505+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[31607]: audit 2024-09-10T18:29:34.264293+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:34 smithi100 bash[31607]: audit 2024-09-10T18:29:34.271465+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:34.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:34 smithi151 bash[22565]: cluster 2024-09-10T18:29:33.207765+0000 mgr.y (mgr.14150) 202 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:34.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:34 smithi151 bash[22565]: audit 2024-09-10T18:29:34.259505+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:34.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:34 smithi151 bash[22565]: audit 2024-09-10T18:29:34.264293+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:34.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:34 smithi151 bash[22565]: audit 2024-09-10T18:29:34.271465+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:36.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:36 smithi100 bash[18597]: cluster 2024-09-10T18:29:35.208348+0000 mgr.y (mgr.14150) 203 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:36 smithi100 bash[31607]: cluster 2024-09-10T18:29:35.208348+0000 mgr.y (mgr.14150) 203 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:36.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:36 smithi151 bash[22565]: cluster 2024-09-10T18:29:35.208348+0000 mgr.y (mgr.14150) 203 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:38.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:38 smithi100 bash[18597]: cluster 2024-09-10T18:29:37.208876+0000 mgr.y (mgr.14150) 204 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:38 smithi100 bash[31607]: cluster 2024-09-10T18:29:37.208876+0000 mgr.y (mgr.14150) 204 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:38.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:38 smithi151 bash[22565]: cluster 2024-09-10T18:29:37.208876+0000 mgr.y (mgr.14150) 204 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:40.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:40 smithi100 bash[18597]: cluster 2024-09-10T18:29:39.209479+0000 mgr.y (mgr.14150) 205 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:40 smithi100 bash[31607]: cluster 2024-09-10T18:29:39.209479+0000 mgr.y (mgr.14150) 205 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:40.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:40 smithi151 bash[22565]: cluster 2024-09-10T18:29:39.209479+0000 mgr.y (mgr.14150) 205 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[18597]: audit 2024-09-10T18:29:41.025459+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[18597]: audit 2024-09-10T18:29:41.034002+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[18597]: audit 2024-09-10T18:29:41.035457+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[18597]: audit 2024-09-10T18:29:41.036742+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[18597]: audit 2024-09-10T18:29:41.045737+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[18597]: cluster 2024-09-10T18:29:41.210002+0000 mgr.y (mgr.14150) 206 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[18597]: audit 2024-09-10T18:29:41.395194+0000 mon.a (mon.0) 421 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[31607]: audit 2024-09-10T18:29:41.025459+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[31607]: audit 2024-09-10T18:29:41.034002+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[31607]: audit 2024-09-10T18:29:41.035457+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:42.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[31607]: audit 2024-09-10T18:29:41.036742+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:42.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[31607]: audit 2024-09-10T18:29:41.045737+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[31607]: cluster 2024-09-10T18:29:41.210002+0000 mgr.y (mgr.14150) 206 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:42.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:42 smithi100 bash[31607]: audit 2024-09-10T18:29:41.395194+0000 mon.a (mon.0) 421 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:29:42.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:42 smithi151 bash[22565]: audit 2024-09-10T18:29:41.025459+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:42 smithi151 bash[22565]: audit 2024-09-10T18:29:41.034002+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:42 smithi151 bash[22565]: audit 2024-09-10T18:29:41.035457+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:42.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:42 smithi151 bash[22565]: audit 2024-09-10T18:29:41.036742+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:42.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:42 smithi151 bash[22565]: audit 2024-09-10T18:29:41.045737+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:42.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:42 smithi151 bash[22565]: cluster 2024-09-10T18:29:41.210002+0000 mgr.y (mgr.14150) 206 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:42.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:42 smithi151 bash[22565]: audit 2024-09-10T18:29:41.395194+0000 mon.a (mon.0) 421 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:29:43.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:43 smithi151 bash[22565]: audit 2024-09-10T18:29:42.061521+0000 mon.a (mon.0) 422 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:29:43.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:43 smithi151 bash[22565]: cluster 2024-09-10T18:29:42.061664+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:29:43.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:43 smithi151 bash[22565]: audit 2024-09-10T18:29:42.061947+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:43.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:43 smithi151 bash[22565]: audit 2024-09-10T18:29:42.062940+0000 mon.a (mon.0) 425 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:29:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[18597]: audit 2024-09-10T18:29:42.061521+0000 mon.a (mon.0) 422 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:29:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[18597]: cluster 2024-09-10T18:29:42.061664+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:29:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[18597]: audit 2024-09-10T18:29:42.061947+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[18597]: audit 2024-09-10T18:29:42.062940+0000 mon.a (mon.0) 425 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:29:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[31607]: audit 2024-09-10T18:29:42.061521+0000 mon.a (mon.0) 422 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:29:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[31607]: cluster 2024-09-10T18:29:42.061664+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:29:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[31607]: audit 2024-09-10T18:29:42.061947+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:43 smithi100 bash[31607]: audit 2024-09-10T18:29:42.062940+0000 mon.a (mon.0) 425 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-09-10T18:29:44.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[18597]: audit 2024-09-10T18:29:43.065410+0000 mon.a (mon.0) 426 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:29:44.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[18597]: cluster 2024-09-10T18:29:43.065669+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-09-10T18:29:44.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[18597]: audit 2024-09-10T18:29:43.066410+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[18597]: audit 2024-09-10T18:29:43.076901+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[18597]: cluster 2024-09-10T18:29:43.210512+0000 mgr.y (mgr.14150) 207 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[31607]: audit 2024-09-10T18:29:43.065410+0000 mon.a (mon.0) 426 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:29:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[31607]: cluster 2024-09-10T18:29:43.065669+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-09-10T18:29:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[31607]: audit 2024-09-10T18:29:43.066410+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[31607]: audit 2024-09-10T18:29:43.076901+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:44 smithi100 bash[31607]: cluster 2024-09-10T18:29:43.210512+0000 mgr.y (mgr.14150) 207 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:44 smithi151 bash[22565]: audit 2024-09-10T18:29:43.065410+0000 mon.a (mon.0) 426 : audit [INF] from='osd.3 v2:172.21.15.100:6813/2754318461' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-09-10T18:29:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:44 smithi151 bash[22565]: cluster 2024-09-10T18:29:43.065669+0000 mon.a (mon.0) 427 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-09-10T18:29:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:44 smithi151 bash[22565]: audit 2024-09-10T18:29:43.066410+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:44 smithi151 bash[22565]: audit 2024-09-10T18:29:43.076901+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:44 smithi151 bash[22565]: cluster 2024-09-10T18:29:43.210512+0000 mgr.y (mgr.14150) 207 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:29:44.686 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 3 on host 'smithi100' 2024-09-10T18:29:44.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.682+0000 7ff60ffff640 1 -- 172.21.15.100:0/3629414130 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff6181076b0 con 0x7ff60406edd0 2024-09-10T18:29:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 >> v2:172.21.15.100:6800/3986579393 conn(0x7ff60406edd0 msgr2=0x7ff604071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 --2- 172.21.15.100:0/3629414130 >> v2:172.21.15.100:6800/3986579393 conn(0x7ff60406edd0 0x7ff604071280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff608009640 tx=0x7ff608035040 comp rx=0 tx=0).stop 2024-09-10T18:29:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fe7c0 msgr2=0x7ff6180fc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:29:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fe7c0 0x7ff6180fc090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff6100028f0 tx=0x7ff610004600 comp rx=0 tx=0).stop 2024-09-10T18:29:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 shutdown_connections 2024-09-10T18:29:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 --2- 172.21.15.100:0/3629414130 >> v2:172.21.15.100:6800/3986579393 conn(0x7ff60406edd0 0x7ff604071280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6180fe7c0 0x7ff6180fc090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6180fde70 0x7ff6180fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 --2- 172.21.15.100:0/3629414130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6180fb050 0x7ff6180fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:29:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 >> 172.21.15.100:0/3629414130 conn(0x7ff6180f6690 msgr2=0x7ff6180f8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:29:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 shutdown_connections 2024-09-10T18:29:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:29:44.686+0000 7ff61dd55640 1 -- 172.21.15.100:0/3629414130 wait complete. 2024-09-10T18:29:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: cluster 2024-09-10T18:29:42.412283+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: cluster 2024-09-10T18:29:42.412422+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: cluster 2024-09-10T18:29:44.079535+0000 mon.a (mon.0) 430 : cluster [INF] osd.3 v2:172.21.15.100:6813/2754318461 boot 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: cluster 2024-09-10T18:29:44.079641+0000 mon.a (mon.0) 431 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: audit 2024-09-10T18:29:44.080408+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: audit 2024-09-10T18:29:44.669639+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: audit 2024-09-10T18:29:44.677118+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[18597]: audit 2024-09-10T18:29:44.683491+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: cluster 2024-09-10T18:29:42.412283+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:29:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: cluster 2024-09-10T18:29:42.412422+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:29:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: cluster 2024-09-10T18:29:44.079535+0000 mon.a (mon.0) 430 : cluster [INF] osd.3 v2:172.21.15.100:6813/2754318461 boot 2024-09-10T18:29:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: cluster 2024-09-10T18:29:44.079641+0000 mon.a (mon.0) 431 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:29:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: audit 2024-09-10T18:29:44.080408+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: audit 2024-09-10T18:29:44.669639+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: audit 2024-09-10T18:29:44.677118+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:45 smithi100 bash[31607]: audit 2024-09-10T18:29:44.683491+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:45.307 DEBUG:teuthology.orchestra.run.smithi100:osd.3> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:29:45.310 INFO:tasks.cephadm:Deploying osd.4 on smithi151 with /dev/vg_nvme/lv_4... 2024-09-10T18:29:45.310 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: cluster 2024-09-10T18:29:42.412283+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: cluster 2024-09-10T18:29:42.412422+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: cluster 2024-09-10T18:29:44.079535+0000 mon.a (mon.0) 430 : cluster [INF] osd.3 v2:172.21.15.100:6813/2754318461 boot 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: cluster 2024-09-10T18:29:44.079641+0000 mon.a (mon.0) 431 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: audit 2024-09-10T18:29:44.080408+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: audit 2024-09-10T18:29:44.669639+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: audit 2024-09-10T18:29:44.677118+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:45 smithi151 bash[22565]: audit 2024-09-10T18:29:44.683491+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:45.507 INFO:journalctl@ceph.osd.3.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:29:46.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:46 smithi151 bash[22565]: cluster 2024-09-10T18:29:45.103165+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-09-10T18:29:46.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:46 smithi151 bash[22565]: cluster 2024-09-10T18:29:45.211091+0000 mgr.y (mgr.14150) 208 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:46.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:46 smithi100 bash[18597]: cluster 2024-09-10T18:29:45.103165+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-09-10T18:29:46.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:46 smithi100 bash[18597]: cluster 2024-09-10T18:29:45.211091+0000 mgr.y (mgr.14150) 208 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:46 smithi100 bash[31607]: cluster 2024-09-10T18:29:45.103165+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-09-10T18:29:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:46 smithi100 bash[31607]: cluster 2024-09-10T18:29:45.211091+0000 mgr.y (mgr.14150) 208 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:48 smithi100 bash[18597]: cluster 2024-09-10T18:29:47.211616+0000 mgr.y (mgr.14150) 209 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:48.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:48 smithi100 bash[31607]: cluster 2024-09-10T18:29:47.211616+0000 mgr.y (mgr.14150) 209 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:48.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:48 smithi151 bash[22565]: cluster 2024-09-10T18:29:47.211616+0000 mgr.y (mgr.14150) 209 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:50.030 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:29:50.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:50 smithi151 bash[22565]: cluster 2024-09-10T18:29:49.212175+0000 mgr.y (mgr.14150) 210 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:50.549 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:50 smithi100 bash[18597]: cluster 2024-09-10T18:29:49.212175+0000 mgr.y (mgr.14150) 210 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:50.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:50 smithi100 bash[31607]: cluster 2024-09-10T18:29:49.212175+0000 mgr.y (mgr.14150) 210 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:52 smithi100 bash[18597]: cluster 2024-09-10T18:29:51.212749+0000 mgr.y (mgr.14150) 211 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:52.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:52 smithi100 bash[31607]: cluster 2024-09-10T18:29:51.212749+0000 mgr.y (mgr.14150) 211 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:52.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:52 smithi151 bash[22565]: cluster 2024-09-10T18:29:51.212749+0000 mgr.y (mgr.14150) 211 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:53.635 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:29:53.836 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:29:53.856 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-09-10T18:29:53.856 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-09-10T18:29:53.856 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0171008 s, 613 MB/s 2024-09-10T18:29:53.857 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-09-10T18:29:54.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:54 smithi151 bash[22565]: cluster 2024-09-10T18:29:53.213357+0000 mgr.y (mgr.14150) 212 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:54.536 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-09-10T18:29:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:54 smithi100 bash[18597]: cluster 2024-09-10T18:29:53.213357+0000 mgr.y (mgr.14150) 212 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:54.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:54 smithi100 bash[31607]: cluster 2024-09-10T18:29:53.213357+0000 mgr.y (mgr.14150) 212 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:56.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: cephadm 2024-09-10T18:29:54.980798+0000 mgr.y (mgr.14150) 213 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:54.989887+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:54.998168+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:54.999830+0000 mon.a (mon.0) 439 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:55.001124+0000 mon.a (mon.0) 440 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:55.002340+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:55.003557+0000 mon.a (mon.0) 442 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: cephadm 2024-09-10T18:29:55.004525+0000 mgr.y (mgr.14150) 214 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 4061M 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:55.011357+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:55.015156+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:55.016427+0000 mon.a (mon.0) 445 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: audit 2024-09-10T18:29:55.024582+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[18597]: cluster 2024-09-10T18:29:55.213899+0000 mgr.y (mgr.14150) 215 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: cephadm 2024-09-10T18:29:54.980798+0000 mgr.y (mgr.14150) 213 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:54.989887+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:54.998168+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:54.999830+0000 mon.a (mon.0) 439 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:55.001124+0000 mon.a (mon.0) 440 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:55.002340+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:55.003557+0000 mon.a (mon.0) 442 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: cephadm 2024-09-10T18:29:55.004525+0000 mgr.y (mgr.14150) 214 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 4061M 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:55.011357+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:55.015156+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:55.016427+0000 mon.a (mon.0) 445 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: audit 2024-09-10T18:29:55.024582+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:55 smithi100 bash[31607]: cluster 2024-09-10T18:29:55.213899+0000 mgr.y (mgr.14150) 215 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:56.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: cephadm 2024-09-10T18:29:54.980798+0000 mgr.y (mgr.14150) 213 : cephadm [INF] Detected new or changed devices on smithi100 2024-09-10T18:29:56.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:54.989887+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:54.998168+0000 mon.a (mon.0) 438 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:54.999830+0000 mon.a (mon.0) 439 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:55.001124+0000 mon.a (mon.0) 440 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:55.002340+0000 mon.a (mon.0) 441 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:55.003557+0000 mon.a (mon.0) 442 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: cephadm 2024-09-10T18:29:55.004525+0000 mgr.y (mgr.14150) 214 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 4061M 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:55.011357+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:55.015156+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:55.016427+0000 mon.a (mon.0) 445 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: audit 2024-09-10T18:29:55.024582+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:29:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:55 smithi151 bash[22565]: cluster 2024-09-10T18:29:55.213899+0000 mgr.y (mgr.14150) 215 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:58.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:29:58 smithi100 bash[18597]: cluster 2024-09-10T18:29:57.214429+0000 mgr.y (mgr.14150) 216 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:58.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:29:58 smithi100 bash[31607]: cluster 2024-09-10T18:29:57.214429+0000 mgr.y (mgr.14150) 216 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:29:58 smithi151 bash[22565]: cluster 2024-09-10T18:29:57.214429+0000 mgr.y (mgr.14150) 216 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:29:59.254 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:30:00.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:00 smithi100 bash[18597]: cluster 2024-09-10T18:29:59.215046+0000 mgr.y (mgr.14150) 217 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:00.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:00 smithi100 bash[18597]: cluster 2024-09-10T18:30:00.000111+0000 mon.a (mon.0) 447 : cluster [INF] overall HEALTH_OK 2024-09-10T18:30:00.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:00 smithi100 bash[31607]: cluster 2024-09-10T18:29:59.215046+0000 mgr.y (mgr.14150) 217 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:00 smithi100 bash[31607]: cluster 2024-09-10T18:30:00.000111+0000 mon.a (mon.0) 447 : cluster [INF] overall HEALTH_OK 2024-09-10T18:30:00.572 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:00 smithi151 bash[22565]: cluster 2024-09-10T18:29:59.215046+0000 mgr.y (mgr.14150) 217 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:00.572 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:00 smithi151 bash[22565]: cluster 2024-09-10T18:30:00.000111+0000 mon.a (mon.0) 447 : cluster [INF] overall HEALTH_OK 2024-09-10T18:30:02.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 -- 172.21.15.151:0/3614015242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd824102430 msgr2=0x7fd8241048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 --2- 172.21.15.151:0/3614015242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd824102430 0x7fd8241048e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd814004950 tx=0x7fd814009da0 comp rx=0 tx=0).stop 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 -- 172.21.15.151:0/3614015242 shutdown_connections 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 --2- 172.21.15.151:0/3614015242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd824102430 0x7fd8241048e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 --2- 172.21.15.151:0/3614015242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8240ff920 0x7fd824101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 --2- 172.21.15.151:0/3614015242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd824066810 0x7fd8240ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 -- 172.21.15.151:0/3614015242 >> 172.21.15.151:0/3614015242 conn(0x7fd8240f6e90 msgr2=0x7fd8240f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 -- 172.21.15.151:0/3614015242 shutdown_connections 2024-09-10T18:30:02.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 -- 172.21.15.151:0/3614015242 wait complete. 2024-09-10T18:30:02.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.153+0000 7fd82a149640 1 Processor -- start 2024-09-10T18:30:02.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 -- start start 2024-09-10T18:30:02.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd824066810 0x7fd82410cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:02.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240ff920 0x7fd82410d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824102430 0x7fd8241140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824114680 con 0x7fd824102430 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8241147f0 con 0x7fd8240ff920 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8241149b0 con 0x7fd824066810 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240ff920 0x7fd82410d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240ff920 0x7fd82410d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52258/0 (socket says 172.21.15.151:52258) 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 -- 172.21.15.151:0/557241579 learned_addr learned my addr 172.21.15.151:0/557241579 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd8237fe640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd824066810 0x7fd82410cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:02.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd823fff640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824102430 0x7fd8241140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 -- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd824066810 msgr2=0x7fd82410cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd824066810 0x7fd82410cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 -- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824102430 msgr2=0x7fd8241140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824102430 0x7fd8241140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 -- 172.21.15.151:0/557241579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814005040 con 0x7fd8240ff920 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd823fff640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824102430 0x7fd8241140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd822ffd640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240ff920 0x7fd82410d050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00ebf0 tx=0x7fd80c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd81bfff640 1 -- 172.21.15.151:0/557241579 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c007c30 con 0x7fd8240ff920 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd81bfff640 1 -- 172.21.15.151:0/557241579 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd80c00f040 con 0x7fd8240ff920 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd824114c30 con 0x7fd8240ff920 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd81bfff640 1 -- 172.21.15.151:0/557241579 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0094f0 con 0x7fd8240ff920 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.157+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd824115150 con 0x7fd8240ff920 2024-09-10T18:30:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.161+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82404bf60 con 0x7fd8240ff920 2024-09-10T18:30:02.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.161+0000 7fd81bfff640 1 -- 172.21.15.151:0/557241579 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7fd80c009860 con 0x7fd8240ff920 2024-09-10T18:30:02.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.161+0000 7fd81bfff640 1 --2- 172.21.15.151:0/557241579 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd81006ee00 0x7fd8100712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:02.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.161+0000 7fd81bfff640 1 -- 172.21.15.151:0/557241579 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 2799+0+0 (secure 0 0 0) 0x7fd80c08d290 con 0x7fd8240ff920 2024-09-10T18:30:02.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.161+0000 7fd8237fe640 1 --2- 172.21.15.151:0/557241579 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd81006ee00 0x7fd8100712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:02.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.161+0000 7fd8237fe640 1 --2- 172.21.15.151:0/557241579 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd81006ee00 0x7fd8100712b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd81c004fd0 tx=0x7fd81c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:30:02.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.169+0000 7fd81bfff640 1 -- 172.21.15.151:0/557241579 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd80c04e2a0 con 0x7fd8240ff920 2024-09-10T18:30:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:02.349+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd824103f20 con 0x7fd81006ee00 2024-09-10T18:30:02.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:02 smithi100 bash[18597]: cluster 2024-09-10T18:30:01.215605+0000 mgr.y (mgr.14150) 218 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:02 smithi100 bash[31607]: cluster 2024-09-10T18:30:01.215605+0000 mgr.y (mgr.14150) 218 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:02 smithi151 bash[22565]: cluster 2024-09-10T18:30:01.215605+0000 mgr.y (mgr.14150) 218 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[18597]: audit 2024-09-10T18:30:02.363978+0000 mgr.y (mgr.14150) 219 : audit [DBG] from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:30:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[18597]: audit 2024-09-10T18:30:02.370981+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:30:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[18597]: audit 2024-09-10T18:30:02.377189+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:30:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[18597]: audit 2024-09-10T18:30:02.379197+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[31607]: audit 2024-09-10T18:30:02.363978+0000 mgr.y (mgr.14150) 219 : audit [DBG] from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:30:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[31607]: audit 2024-09-10T18:30:02.370981+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:30:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[31607]: audit 2024-09-10T18:30:02.377189+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:30:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:03 smithi100 bash[31607]: audit 2024-09-10T18:30:02.379197+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:03 smithi151 bash[22565]: audit 2024-09-10T18:30:02.363978+0000 mgr.y (mgr.14150) 219 : audit [DBG] from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:30:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:03 smithi151 bash[22565]: audit 2024-09-10T18:30:02.370981+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:30:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:03 smithi151 bash[22565]: audit 2024-09-10T18:30:02.377189+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:30:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:03 smithi151 bash[22565]: audit 2024-09-10T18:30:02.379197+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:04 smithi100 bash[18597]: cluster 2024-09-10T18:30:03.216149+0000 mgr.y (mgr.14150) 220 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:04.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:04 smithi100 bash[31607]: cluster 2024-09-10T18:30:03.216149+0000 mgr.y (mgr.14150) 220 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:04 smithi151 bash[22565]: cluster 2024-09-10T18:30:03.216149+0000 mgr.y (mgr.14150) 220 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:06 smithi151 bash[22565]: cluster 2024-09-10T18:30:05.216663+0000 mgr.y (mgr.14150) 221 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:06 smithi100 bash[18597]: cluster 2024-09-10T18:30:05.216663+0000 mgr.y (mgr.14150) 221 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:06.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:06 smithi100 bash[31607]: cluster 2024-09-10T18:30:05.216663+0000 mgr.y (mgr.14150) 221 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:08.588 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:08 smithi151 bash[22565]: cluster 2024-09-10T18:30:07.217212+0000 mgr.y (mgr.14150) 222 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:08 smithi100 bash[18597]: cluster 2024-09-10T18:30:07.217212+0000 mgr.y (mgr.14150) 222 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:08.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:08 smithi100 bash[31607]: cluster 2024-09-10T18:30:07.217212+0000 mgr.y (mgr.14150) 222 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:10.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:10 smithi151 bash[22565]: cluster 2024-09-10T18:30:09.217818+0000 mgr.y (mgr.14150) 223 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:10 smithi100 bash[18597]: cluster 2024-09-10T18:30:09.217818+0000 mgr.y (mgr.14150) 223 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:10.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:10 smithi100 bash[31607]: cluster 2024-09-10T18:30:09.217818+0000 mgr.y (mgr.14150) 223 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:11 smithi151 bash[22565]: audit 2024-09-10T18:30:11.023447+0000 mon.b (mon.1) 10 : audit [INF] from='client.? 172.21.15.151:0/1263507825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]: dispatch 2024-09-10T18:30:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:11 smithi151 bash[22565]: audit 2024-09-10T18:30:11.033742+0000 mon.a (mon.0) 451 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]: dispatch 2024-09-10T18:30:11.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:11 smithi151 bash[22565]: audit 2024-09-10T18:30:11.042915+0000 mon.a (mon.0) 452 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]': finished 2024-09-10T18:30:11.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:11 smithi151 bash[22565]: cluster 2024-09-10T18:30:11.043117+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:30:11.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:11 smithi151 bash[22565]: audit 2024-09-10T18:30:11.043429+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[18597]: audit 2024-09-10T18:30:11.023447+0000 mon.b (mon.1) 10 : audit [INF] from='client.? 172.21.15.151:0/1263507825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]: dispatch 2024-09-10T18:30:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[18597]: audit 2024-09-10T18:30:11.033742+0000 mon.a (mon.0) 451 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]: dispatch 2024-09-10T18:30:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[18597]: audit 2024-09-10T18:30:11.042915+0000 mon.a (mon.0) 452 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]': finished 2024-09-10T18:30:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[18597]: cluster 2024-09-10T18:30:11.043117+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:30:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[18597]: audit 2024-09-10T18:30:11.043429+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[31607]: audit 2024-09-10T18:30:11.023447+0000 mon.b (mon.1) 10 : audit [INF] from='client.? 172.21.15.151:0/1263507825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]: dispatch 2024-09-10T18:30:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[31607]: audit 2024-09-10T18:30:11.033742+0000 mon.a (mon.0) 451 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]: dispatch 2024-09-10T18:30:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[31607]: audit 2024-09-10T18:30:11.042915+0000 mon.a (mon.0) 452 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80851ca1-0c33-4ac6-80b5-92bbcaa3dacb"}]': finished 2024-09-10T18:30:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[31607]: cluster 2024-09-10T18:30:11.043117+0000 mon.a (mon.0) 453 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:30:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:11 smithi100 bash[31607]: audit 2024-09-10T18:30:11.043429+0000 mon.a (mon.0) 454 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:12.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:12 smithi151 bash[22565]: cluster 2024-09-10T18:30:11.218336+0000 mgr.y (mgr.14150) 224 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:12.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:12 smithi151 bash[22565]: audit 2024-09-10T18:30:11.655621+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.151:0/263148342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:30:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:12 smithi100 bash[18597]: cluster 2024-09-10T18:30:11.218336+0000 mgr.y (mgr.14150) 224 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:12 smithi100 bash[18597]: audit 2024-09-10T18:30:11.655621+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.151:0/263148342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:30:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:12 smithi100 bash[31607]: cluster 2024-09-10T18:30:11.218336+0000 mgr.y (mgr.14150) 224 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:12 smithi100 bash[31607]: audit 2024-09-10T18:30:11.655621+0000 mon.b (mon.1) 11 : audit [DBG] from='client.? 172.21.15.151:0/263148342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:30:14.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:14 smithi151 bash[22565]: cluster 2024-09-10T18:30:13.218881+0000 mgr.y (mgr.14150) 225 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:14.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:14 smithi100 bash[18597]: cluster 2024-09-10T18:30:13.218881+0000 mgr.y (mgr.14150) 225 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:14.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:14 smithi100 bash[31607]: cluster 2024-09-10T18:30:13.218881+0000 mgr.y (mgr.14150) 225 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:16.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:16 smithi151 bash[22565]: cluster 2024-09-10T18:30:15.219391+0000 mgr.y (mgr.14150) 226 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:16 smithi100 bash[18597]: cluster 2024-09-10T18:30:15.219391+0000 mgr.y (mgr.14150) 226 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:16 smithi100 bash[31607]: cluster 2024-09-10T18:30:15.219391+0000 mgr.y (mgr.14150) 226 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:18.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:18 smithi151 bash[22565]: cluster 2024-09-10T18:30:17.219943+0000 mgr.y (mgr.14150) 227 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:18 smithi100 bash[31607]: cluster 2024-09-10T18:30:17.219943+0000 mgr.y (mgr.14150) 227 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:18.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:18 smithi100 bash[18597]: cluster 2024-09-10T18:30:17.219943+0000 mgr.y (mgr.14150) 227 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:20.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:20 smithi151 bash[22565]: cluster 2024-09-10T18:30:19.220499+0000 mgr.y (mgr.14150) 228 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:20.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:20 smithi100 bash[31607]: cluster 2024-09-10T18:30:19.220499+0000 mgr.y (mgr.14150) 228 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:20.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:20 smithi100 bash[18597]: cluster 2024-09-10T18:30:19.220499+0000 mgr.y (mgr.14150) 228 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:22.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:22 smithi151 bash[22565]: cluster 2024-09-10T18:30:21.221111+0000 mgr.y (mgr.14150) 229 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:22 smithi100 bash[18597]: cluster 2024-09-10T18:30:21.221111+0000 mgr.y (mgr.14150) 229 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:22.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:22 smithi100 bash[31607]: cluster 2024-09-10T18:30:21.221111+0000 mgr.y (mgr.14150) 229 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:24.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:24 smithi151 bash[22565]: cluster 2024-09-10T18:30:23.221679+0000 mgr.y (mgr.14150) 230 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:24.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:24 smithi151 bash[22565]: audit 2024-09-10T18:30:23.816586+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:30:24.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:24 smithi151 bash[22565]: audit 2024-09-10T18:30:23.818536+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:24.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:24 smithi151 bash[22565]: cephadm 2024-09-10T18:30:23.820178+0000 mgr.y (mgr.14150) 231 : cephadm [INF] Deploying daemon osd.4 on smithi151 2024-09-10T18:30:24.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[18597]: cluster 2024-09-10T18:30:23.221679+0000 mgr.y (mgr.14150) 230 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:24.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[18597]: audit 2024-09-10T18:30:23.816586+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:30:24.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[18597]: audit 2024-09-10T18:30:23.818536+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:24.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[18597]: cephadm 2024-09-10T18:30:23.820178+0000 mgr.y (mgr.14150) 231 : cephadm [INF] Deploying daemon osd.4 on smithi151 2024-09-10T18:30:24.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[31607]: cluster 2024-09-10T18:30:23.221679+0000 mgr.y (mgr.14150) 230 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:24.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[31607]: audit 2024-09-10T18:30:23.816586+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:30:24.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[31607]: audit 2024-09-10T18:30:23.818536+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:24.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:24 smithi100 bash[31607]: cephadm 2024-09-10T18:30:23.820178+0000 mgr.y (mgr.14150) 231 : cephadm [INF] Deploying daemon osd.4 on smithi151 2024-09-10T18:30:26.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:26 smithi151 bash[22565]: cluster 2024-09-10T18:30:25.222227+0000 mgr.y (mgr.14150) 232 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:26 smithi100 bash[18597]: cluster 2024-09-10T18:30:25.222227+0000 mgr.y (mgr.14150) 232 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:26.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:26 smithi100 bash[31607]: cluster 2024-09-10T18:30:25.222227+0000 mgr.y (mgr.14150) 232 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:28.591 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:28 smithi151 bash[22565]: cluster 2024-09-10T18:30:27.222755+0000 mgr.y (mgr.14150) 233 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:28 smithi100 bash[18597]: cluster 2024-09-10T18:30:27.222755+0000 mgr.y (mgr.14150) 233 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:28 smithi100 bash[31607]: cluster 2024-09-10T18:30:27.222755+0000 mgr.y (mgr.14150) 233 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:29.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:29 smithi151 bash[22565]: audit 2024-09-10T18:30:28.489924+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:29.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:29 smithi151 bash[22565]: audit 2024-09-10T18:30:28.498843+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:29.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:29 smithi151 bash[22565]: audit 2024-09-10T18:30:28.508103+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:29 smithi100 bash[18597]: audit 2024-09-10T18:30:28.489924+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:29 smithi100 bash[18597]: audit 2024-09-10T18:30:28.498843+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:29 smithi100 bash[18597]: audit 2024-09-10T18:30:28.508103+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:29 smithi100 bash[31607]: audit 2024-09-10T18:30:28.489924+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:29 smithi100 bash[31607]: audit 2024-09-10T18:30:28.498843+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:29 smithi100 bash[31607]: audit 2024-09-10T18:30:28.508103+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:30.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:30 smithi151 bash[22565]: cluster 2024-09-10T18:30:29.223298+0000 mgr.y (mgr.14150) 234 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:30 smithi100 bash[18597]: cluster 2024-09-10T18:30:29.223298+0000 mgr.y (mgr.14150) 234 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:30.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:30 smithi100 bash[31607]: cluster 2024-09-10T18:30:29.223298+0000 mgr.y (mgr.14150) 234 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:32.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:32 smithi151 bash[22565]: cluster 2024-09-10T18:30:31.223834+0000 mgr.y (mgr.14150) 235 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:32 smithi100 bash[18597]: cluster 2024-09-10T18:30:31.223834+0000 mgr.y (mgr.14150) 235 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:32.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:32 smithi100 bash[31607]: cluster 2024-09-10T18:30:31.223834+0000 mgr.y (mgr.14150) 235 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:34.760 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:34 smithi151 bash[22565]: cluster 2024-09-10T18:30:33.224380+0000 mgr.y (mgr.14150) 236 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:34 smithi100 bash[18597]: cluster 2024-09-10T18:30:33.224380+0000 mgr.y (mgr.14150) 236 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:34 smithi100 bash[31607]: cluster 2024-09-10T18:30:33.224380+0000 mgr.y (mgr.14150) 236 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: audit 2024-09-10T18:30:34.983236+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: audit 2024-09-10T18:30:34.991466+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: cluster 2024-09-10T18:30:35.224901+0000 mgr.y (mgr.14150) 237 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: audit 2024-09-10T18:30:35.282913+0000 mon.b (mon.1) 12 : audit [INF] from='osd.4 v2:172.21.15.151:6800/2989412742' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:30:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: audit 2024-09-10T18:30:35.293596+0000 mon.a (mon.0) 462 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:30:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: audit 2024-09-10T18:30:35.380963+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: audit 2024-09-10T18:30:35.382996+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:30:36.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:35 smithi151 bash[22565]: audit 2024-09-10T18:30:35.393394+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: audit 2024-09-10T18:30:34.983236+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: audit 2024-09-10T18:30:34.991466+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: cluster 2024-09-10T18:30:35.224901+0000 mgr.y (mgr.14150) 237 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: audit 2024-09-10T18:30:35.282913+0000 mon.b (mon.1) 12 : audit [INF] from='osd.4 v2:172.21.15.151:6800/2989412742' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:30:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: audit 2024-09-10T18:30:35.293596+0000 mon.a (mon.0) 462 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: audit 2024-09-10T18:30:35.380963+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: audit 2024-09-10T18:30:35.382996+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[18597]: audit 2024-09-10T18:30:35.393394+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: audit 2024-09-10T18:30:34.983236+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: audit 2024-09-10T18:30:34.991466+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: cluster 2024-09-10T18:30:35.224901+0000 mgr.y (mgr.14150) 237 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: audit 2024-09-10T18:30:35.282913+0000 mon.b (mon.1) 12 : audit [INF] from='osd.4 v2:172.21.15.151:6800/2989412742' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: audit 2024-09-10T18:30:35.293596+0000 mon.a (mon.0) 462 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: audit 2024-09-10T18:30:35.380963+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: audit 2024-09-10T18:30:35.382996+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:30:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:35 smithi100 bash[31607]: audit 2024-09-10T18:30:35.393394+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[31607]: audit 2024-09-10T18:30:35.997763+0000 mon.b (mon.1) 13 : audit [INF] from='osd.4 v2:172.21.15.151:6800/2989412742' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:30:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[31607]: audit 2024-09-10T18:30:36.006284+0000 mon.a (mon.0) 466 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:30:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[31607]: cluster 2024-09-10T18:30:36.006547+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:30:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[31607]: audit 2024-09-10T18:30:36.006789+0000 mon.a (mon.0) 468 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[31607]: audit 2024-09-10T18:30:36.008392+0000 mon.a (mon.0) 469 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:30:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[18597]: audit 2024-09-10T18:30:35.997763+0000 mon.b (mon.1) 13 : audit [INF] from='osd.4 v2:172.21.15.151:6800/2989412742' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:30:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[18597]: audit 2024-09-10T18:30:36.006284+0000 mon.a (mon.0) 466 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:30:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[18597]: cluster 2024-09-10T18:30:36.006547+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:30:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[18597]: audit 2024-09-10T18:30:36.006789+0000 mon.a (mon.0) 468 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:37 smithi100 bash[18597]: audit 2024-09-10T18:30:36.008392+0000 mon.a (mon.0) 469 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:30:37.348 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:37 smithi151 bash[22565]: audit 2024-09-10T18:30:35.997763+0000 mon.b (mon.1) 13 : audit [INF] from='osd.4 v2:172.21.15.151:6800/2989412742' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:30:37.348 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:37 smithi151 bash[22565]: audit 2024-09-10T18:30:36.006284+0000 mon.a (mon.0) 466 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:30:37.348 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:37 smithi151 bash[22565]: cluster 2024-09-10T18:30:36.006547+0000 mon.a (mon.0) 467 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:30:37.348 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:37 smithi151 bash[22565]: audit 2024-09-10T18:30:36.006789+0000 mon.a (mon.0) 468 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:37.348 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:37 smithi151 bash[22565]: audit 2024-09-10T18:30:36.008392+0000 mon.a (mon.0) 469 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: audit 2024-09-10T18:30:37.013044+0000 mon.a (mon.0) 470 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: cluster 2024-09-10T18:30:37.013213+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: audit 2024-09-10T18:30:37.013956+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: audit 2024-09-10T18:30:37.024745+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: cluster 2024-09-10T18:30:37.225466+0000 mgr.y (mgr.14150) 238 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: cluster 2024-09-10T18:30:38.011187+0000 mon.a (mon.0) 474 : cluster [INF] osd.4 v2:172.21.15.151:6800/2989412742 boot 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: cluster 2024-09-10T18:30:38.011324+0000 mon.a (mon.0) 475 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:30:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:38 smithi151 bash[22565]: audit 2024-09-10T18:30:38.011754+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: audit 2024-09-10T18:30:37.013044+0000 mon.a (mon.0) 470 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:30:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: cluster 2024-09-10T18:30:37.013213+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-09-10T18:30:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: audit 2024-09-10T18:30:37.013956+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: audit 2024-09-10T18:30:37.024745+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: cluster 2024-09-10T18:30:37.225466+0000 mgr.y (mgr.14150) 238 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: cluster 2024-09-10T18:30:38.011187+0000 mon.a (mon.0) 474 : cluster [INF] osd.4 v2:172.21.15.151:6800/2989412742 boot 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: cluster 2024-09-10T18:30:38.011324+0000 mon.a (mon.0) 475 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[31607]: audit 2024-09-10T18:30:38.011754+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: audit 2024-09-10T18:30:37.013044+0000 mon.a (mon.0) 470 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: cluster 2024-09-10T18:30:37.013213+0000 mon.a (mon.0) 471 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: audit 2024-09-10T18:30:37.013956+0000 mon.a (mon.0) 472 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: audit 2024-09-10T18:30:37.024745+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: cluster 2024-09-10T18:30:37.225466+0000 mgr.y (mgr.14150) 238 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: cluster 2024-09-10T18:30:38.011187+0000 mon.a (mon.0) 474 : cluster [INF] osd.4 v2:172.21.15.151:6800/2989412742 boot 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: cluster 2024-09-10T18:30:38.011324+0000 mon.a (mon.0) 475 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:30:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:38 smithi100 bash[18597]: audit 2024-09-10T18:30:38.011754+0000 mon.a (mon.0) 476 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:30:38.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.456+0000 7fd81bfff640 1 -- 172.21.15.151:0/557241579 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd824103f20 con 0x7fd81006ee00 2024-09-10T18:30:38.471 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 4 on host 'smithi151' 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd81006ee00 msgr2=0x7fd8100712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 --2- 172.21.15.151:0/557241579 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd81006ee00 0x7fd8100712b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd81c004fd0 tx=0x7fd81c00b040 comp rx=0 tx=0).stop 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240ff920 msgr2=0x7fd82410d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240ff920 0x7fd82410d050 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00ebf0 tx=0x7fd80c00b040 comp rx=0 tx=0).stop 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 shutdown_connections 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 --2- 172.21.15.151:0/557241579 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd81006ee00 0x7fd8100712b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd824102430 0x7fd8241140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:38.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240ff920 0x7fd82410d050 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:38.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 --2- 172.21.15.151:0/557241579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd824066810 0x7fd82410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:38.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.460+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 >> 172.21.15.151:0/557241579 conn(0x7fd8240f6e90 msgr2=0x7fd824102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:30:38.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.464+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 shutdown_connections 2024-09-10T18:30:38.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:38.464+0000 7fd82a149640 1 -- 172.21.15.151:0/557241579 wait complete. 2024-09-10T18:30:39.108 DEBUG:teuthology.orchestra.run.smithi151:osd.4> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:30:39.111 INFO:tasks.cephadm:Deploying osd.5 on smithi151 with /dev/vg_nvme/lv_3... 2024-09-10T18:30:39.111 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:30:39.257 INFO:journalctl@ceph.osd.4.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:30:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[18597]: cluster 2024-09-10T18:30:36.277520+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:30:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[18597]: cluster 2024-09-10T18:30:36.277599+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:30:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[18597]: audit 2024-09-10T18:30:38.451596+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[18597]: audit 2024-09-10T18:30:38.461109+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[18597]: audit 2024-09-10T18:30:38.467930+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[18597]: cluster 2024-09-10T18:30:39.013904+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:30:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[31607]: cluster 2024-09-10T18:30:36.277520+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:30:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[31607]: cluster 2024-09-10T18:30:36.277599+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:30:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[31607]: audit 2024-09-10T18:30:38.451596+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[31607]: audit 2024-09-10T18:30:38.461109+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[31607]: audit 2024-09-10T18:30:38.467930+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:39 smithi100 bash[31607]: cluster 2024-09-10T18:30:39.013904+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:30:39.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:39 smithi151 bash[22565]: cluster 2024-09-10T18:30:36.277520+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:30:39.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:39 smithi151 bash[22565]: cluster 2024-09-10T18:30:36.277599+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:30:39.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:39 smithi151 bash[22565]: audit 2024-09-10T18:30:38.451596+0000 mon.a (mon.0) 477 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:30:39.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:39 smithi151 bash[22565]: audit 2024-09-10T18:30:38.461109+0000 mon.a (mon.0) 478 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:39.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:39 smithi151 bash[22565]: audit 2024-09-10T18:30:38.467930+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:39.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:39 smithi151 bash[22565]: cluster 2024-09-10T18:30:39.013904+0000 mon.a (mon.0) 480 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:30:40.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:40 smithi100 bash[18597]: cluster 2024-09-10T18:30:39.225959+0000 mgr.y (mgr.14150) 239 : cluster [DBG] pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:40.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:40 smithi100 bash[18597]: cluster 2024-09-10T18:30:40.015587+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:30:40.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:40 smithi100 bash[31607]: cluster 2024-09-10T18:30:39.225959+0000 mgr.y (mgr.14150) 239 : cluster [DBG] pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:40 smithi100 bash[31607]: cluster 2024-09-10T18:30:40.015587+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:30:40.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:40 smithi151 bash[22565]: cluster 2024-09-10T18:30:39.225959+0000 mgr.y (mgr.14150) 239 : cluster [DBG] pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:40.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:40 smithi151 bash[22565]: cluster 2024-09-10T18:30:40.015587+0000 mon.a (mon.0) 481 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:30:42.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:42 smithi100 bash[18597]: cluster 2024-09-10T18:30:41.226510+0000 mgr.y (mgr.14150) 240 : cluster [DBG] pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:42.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:42 smithi100 bash[31607]: cluster 2024-09-10T18:30:41.226510+0000 mgr.y (mgr.14150) 240 : cluster [DBG] pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:42.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:42 smithi151 bash[22565]: cluster 2024-09-10T18:30:41.226510+0000 mgr.y (mgr.14150) 240 : cluster [DBG] pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:43.217 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:30:44.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:44 smithi100 bash[18597]: cluster 2024-09-10T18:30:43.227116+0000 mgr.y (mgr.14150) 241 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:30:44.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:44 smithi100 bash[31607]: cluster 2024-09-10T18:30:43.227116+0000 mgr.y (mgr.14150) 241 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:30:44.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:44 smithi151 bash[22565]: cluster 2024-09-10T18:30:43.227116+0000 mgr.y (mgr.14150) 241 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:30:46.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:46 smithi100 bash[18597]: cluster 2024-09-10T18:30:45.227689+0000 mgr.y (mgr.14150) 242 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:30:46.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:46 smithi100 bash[31607]: cluster 2024-09-10T18:30:45.227689+0000 mgr.y (mgr.14150) 242 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:30:46.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:46 smithi151 bash[22565]: cluster 2024-09-10T18:30:45.227689+0000 mgr.y (mgr.14150) 242 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T18:30:48.268 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:30:48.471 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:30:48.492 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-09-10T18:30:48.492 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-09-10T18:30:48.492 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0170616 s, 615 MB/s 2024-09-10T18:30:48.492 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-09-10T18:30:48.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:48 smithi151 bash[22565]: cluster 2024-09-10T18:30:47.228257+0000 mgr.y (mgr.14150) 243 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T18:30:48.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:48 smithi100 bash[18597]: cluster 2024-09-10T18:30:47.228257+0000 mgr.y (mgr.14150) 243 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T18:30:48.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:48 smithi100 bash[31607]: cluster 2024-09-10T18:30:47.228257+0000 mgr.y (mgr.14150) 243 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T18:30:49.203 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-09-10T18:30:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: cluster 2024-09-10T18:30:49.228856+0000 mgr.y (mgr.14150) 244 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:30:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: audit 2024-09-10T18:30:50.031207+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: audit 2024-09-10T18:30:50.038043+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: audit 2024-09-10T18:30:50.039499+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:30:50.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: audit 2024-09-10T18:30:50.047731+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: audit 2024-09-10T18:30:50.049435+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:50.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: audit 2024-09-10T18:30:50.050650+0000 mon.a (mon.0) 487 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:30:50.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:50 smithi151 bash[22565]: audit 2024-09-10T18:30:50.058753+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: cluster 2024-09-10T18:30:49.228856+0000 mgr.y (mgr.14150) 244 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:30:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: audit 2024-09-10T18:30:50.031207+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: audit 2024-09-10T18:30:50.038043+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: audit 2024-09-10T18:30:50.039499+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:30:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: audit 2024-09-10T18:30:50.047731+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: audit 2024-09-10T18:30:50.049435+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: audit 2024-09-10T18:30:50.050650+0000 mon.a (mon.0) 487 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[18597]: audit 2024-09-10T18:30:50.058753+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: cluster 2024-09-10T18:30:49.228856+0000 mgr.y (mgr.14150) 244 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: audit 2024-09-10T18:30:50.031207+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: audit 2024-09-10T18:30:50.038043+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: audit 2024-09-10T18:30:50.039499+0000 mon.a (mon.0) 484 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: audit 2024-09-10T18:30:50.047731+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: audit 2024-09-10T18:30:50.049435+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: audit 2024-09-10T18:30:50.050650+0000 mon.a (mon.0) 487 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:30:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:50 smithi100 bash[31607]: audit 2024-09-10T18:30:50.058753+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:30:51.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:51 smithi151 bash[22565]: cephadm 2024-09-10T18:30:50.022730+0000 mgr.y (mgr.14150) 245 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:30:51.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:51 smithi151 bash[22565]: cephadm 2024-09-10T18:30:50.040571+0000 mgr.y (mgr.14150) 246 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 17271M 2024-09-10T18:30:51.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:51 smithi100 bash[18597]: cephadm 2024-09-10T18:30:50.022730+0000 mgr.y (mgr.14150) 245 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:30:51.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:51 smithi100 bash[18597]: cephadm 2024-09-10T18:30:50.040571+0000 mgr.y (mgr.14150) 246 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 17271M 2024-09-10T18:30:51.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:51 smithi100 bash[31607]: cephadm 2024-09-10T18:30:50.022730+0000 mgr.y (mgr.14150) 245 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:30:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:51 smithi100 bash[31607]: cephadm 2024-09-10T18:30:50.040571+0000 mgr.y (mgr.14150) 246 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 17271M 2024-09-10T18:30:52.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:52 smithi151 bash[22565]: cluster 2024-09-10T18:30:51.229502+0000 mgr.y (mgr.14150) 247 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T18:30:52.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:52 smithi100 bash[18597]: cluster 2024-09-10T18:30:51.229502+0000 mgr.y (mgr.14150) 247 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T18:30:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:52 smithi100 bash[31607]: cluster 2024-09-10T18:30:51.229502+0000 mgr.y (mgr.14150) 247 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T18:30:53.945 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:30:54.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:54 smithi151 bash[22565]: cluster 2024-09-10T18:30:53.230108+0000 mgr.y (mgr.14150) 248 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:30:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:54 smithi100 bash[18597]: cluster 2024-09-10T18:30:53.230108+0000 mgr.y (mgr.14150) 248 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:30:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:54 smithi100 bash[31607]: cluster 2024-09-10T18:30:53.230108+0000 mgr.y (mgr.14150) 248 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:30:56.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:56 smithi151 bash[22565]: cluster 2024-09-10T18:30:55.230650+0000 mgr.y (mgr.14150) 249 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:56 smithi100 bash[18597]: cluster 2024-09-10T18:30:55.230650+0000 mgr.y (mgr.14150) 249 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:56.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:56 smithi100 bash[31607]: cluster 2024-09-10T18:30:55.230650+0000 mgr.y (mgr.14150) 249 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 -- 172.21.15.151:0/2209333141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 msgr2=0x7f999c0fec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 --2- 172.21.15.151:0/2209333141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c0fec90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f998c009cd0 tx=0x7f998c004d10 comp rx=0 tx=0).stop 2024-09-10T18:30:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 -- 172.21.15.151:0/2209333141 shutdown_connections 2024-09-10T18:30:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 --2- 172.21.15.151:0/2209333141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f999c100500 0x7f999c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 --2- 172.21.15.151:0/2209333141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999c0ffb50 0x7f999c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 --2- 172.21.15.151:0/2209333141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c0fec90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 -- 172.21.15.151:0/2209333141 >> 172.21.15.151:0/2209333141 conn(0x7f999c0fa060 msgr2=0x7f999c0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:30:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 -- 172.21.15.151:0/2209333141 shutdown_connections 2024-09-10T18:30:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.228+0000 7f99a1a47640 1 -- 172.21.15.151:0/2209333141 wait complete. 2024-09-10T18:30:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 Processor -- start 2024-09-10T18:30:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 -- start start 2024-09-10T18:30:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c10ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999c0ffb50 0x7f999c10f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f999c100500 0x7f999c116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c1168f0 con 0x7f999c0ffb50 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c116a60 con 0x7f999c0fe880 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999c116c20 con 0x7f999c100500 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c10ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c10ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35668/0 (socket says 172.21.15.151:35668) 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 -- 172.21.15.151:0/661076741 learned_addr learned my addr 172.21.15.151:0/661076741 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:30:57.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f999bfff640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999c0ffb50 0x7f999c10f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1246640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f999c100500 0x7f999c116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 -- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f999c100500 msgr2=0x7f999c116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f999c100500 0x7f999c116320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 -- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999c0ffb50 msgr2=0x7f999c10f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999c0ffb50 0x7f999c10f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 -- 172.21.15.151:0/661076741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c0098b0 con 0x7f999c0fe880 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a0a45640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c10ee10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f998c004750 tx=0x7f998c004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99997fa640 1 -- 172.21.15.151:0/661076741 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c038070 con 0x7f999c0fe880 2024-09-10T18:30:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99997fa640 1 -- 172.21.15.151:0/661076741 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f998c033da0 con 0x7f999c0fe880 2024-09-10T18:30:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c116ea0 con 0x7f999c0fe880 2024-09-10T18:30:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99997fa640 1 -- 172.21.15.151:0/661076741 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c03cb10 con 0x7f999c0fe880 2024-09-10T18:30:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.232+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c117300 con 0x7f999c0fe880 2024-09-10T18:30:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.236+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999c04bf60 con 0x7f999c0fe880 2024-09-10T18:30:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.236+0000 7f99997fa640 1 -- 172.21.15.151:0/661076741 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7f998c030070 con 0x7f999c0fe880 2024-09-10T18:30:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.236+0000 7f99997fa640 1 --2- 172.21.15.151:0/661076741 >> v2:172.21.15.100:6800/3986579393 conn(0x7f998806edd0 0x7f9988071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:30:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.236+0000 7f99997fa640 1 -- 172.21.15.151:0/661076741 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 3201+0+0 (secure 0 0 0) 0x7f998c0adc30 con 0x7f999c0fe880 2024-09-10T18:30:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.236+0000 7f999bfff640 1 --2- 172.21.15.151:0/661076741 >> v2:172.21.15.100:6800/3986579393 conn(0x7f998806edd0 0x7f9988071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:30:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.236+0000 7f999bfff640 1 --2- 172.21.15.151:0/661076741 >> v2:172.21.15.100:6800/3986579393 conn(0x7f998806edd0 0x7f9988071280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9984004eb0 tx=0x7f998400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:30:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.244+0000 7f99997fa640 1 -- 172.21.15.151:0/661076741 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f998c07b6d0 con 0x7f999c0fe880 2024-09-10T18:30:57.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:30:57.428+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f999c05db50 con 0x7f998806edd0 2024-09-10T18:30:58.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:58 smithi151 bash[22565]: cluster 2024-09-10T18:30:57.231185+0000 mgr.y (mgr.14150) 250 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:58.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:58 smithi151 bash[22565]: audit 2024-09-10T18:30:57.443195+0000 mgr.y (mgr.14150) 251 : audit [DBG] from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:30:58.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:58 smithi151 bash[22565]: audit 2024-09-10T18:30:57.449599+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:30:58.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:58 smithi151 bash[22565]: audit 2024-09-10T18:30:57.455741+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:30:58.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:30:58 smithi151 bash[22565]: audit 2024-09-10T18:30:57.457277+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[18597]: cluster 2024-09-10T18:30:57.231185+0000 mgr.y (mgr.14150) 250 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[18597]: audit 2024-09-10T18:30:57.443195+0000 mgr.y (mgr.14150) 251 : audit [DBG] from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:30:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[18597]: audit 2024-09-10T18:30:57.449599+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:30:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[18597]: audit 2024-09-10T18:30:57.455741+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:30:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[18597]: audit 2024-09-10T18:30:57.457277+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:30:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[31607]: cluster 2024-09-10T18:30:57.231185+0000 mgr.y (mgr.14150) 250 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:30:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[31607]: audit 2024-09-10T18:30:57.443195+0000 mgr.y (mgr.14150) 251 : audit [DBG] from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:30:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[31607]: audit 2024-09-10T18:30:57.449599+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:30:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[31607]: audit 2024-09-10T18:30:57.455741+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:30:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:30:58 smithi100 bash[31607]: audit 2024-09-10T18:30:57.457277+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:00 smithi151 bash[22565]: cluster 2024-09-10T18:30:59.231785+0000 mgr.y (mgr.14150) 252 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:00.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:00 smithi100 bash[18597]: cluster 2024-09-10T18:30:59.231785+0000 mgr.y (mgr.14150) 252 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:00.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:00 smithi100 bash[31607]: cluster 2024-09-10T18:30:59.231785+0000 mgr.y (mgr.14150) 252 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:02.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:02 smithi151 bash[22565]: cluster 2024-09-10T18:31:01.232324+0000 mgr.y (mgr.14150) 253 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:02 smithi100 bash[18597]: cluster 2024-09-10T18:31:01.232324+0000 mgr.y (mgr.14150) 253 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:02.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:02 smithi100 bash[31607]: cluster 2024-09-10T18:31:01.232324+0000 mgr.y (mgr.14150) 253 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:04.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:04 smithi151 bash[22565]: cluster 2024-09-10T18:31:03.232905+0000 mgr.y (mgr.14150) 254 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:04.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:04 smithi100 bash[18597]: cluster 2024-09-10T18:31:03.232905+0000 mgr.y (mgr.14150) 254 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:04.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:04 smithi100 bash[31607]: cluster 2024-09-10T18:31:03.232905+0000 mgr.y (mgr.14150) 254 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:06.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:06 smithi151 bash[22565]: cluster 2024-09-10T18:31:05.233499+0000 mgr.y (mgr.14150) 255 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:06.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:06 smithi151 bash[22565]: audit 2024-09-10T18:31:06.068349+0000 mon.b (mon.1) 14 : audit [INF] from='client.? 172.21.15.151:0/2976399911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]: dispatch 2024-09-10T18:31:06.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:06 smithi151 bash[22565]: audit 2024-09-10T18:31:06.079478+0000 mon.a (mon.0) 492 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]: dispatch 2024-09-10T18:31:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:06 smithi151 bash[22565]: audit 2024-09-10T18:31:06.093365+0000 mon.a (mon.0) 493 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]': finished 2024-09-10T18:31:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:06 smithi151 bash[22565]: cluster 2024-09-10T18:31:06.093620+0000 mon.a (mon.0) 494 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:31:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:06 smithi151 bash[22565]: audit 2024-09-10T18:31:06.093792+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[18597]: cluster 2024-09-10T18:31:05.233499+0000 mgr.y (mgr.14150) 255 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[18597]: audit 2024-09-10T18:31:06.068349+0000 mon.b (mon.1) 14 : audit [INF] from='client.? 172.21.15.151:0/2976399911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]: dispatch 2024-09-10T18:31:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[18597]: audit 2024-09-10T18:31:06.079478+0000 mon.a (mon.0) 492 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]: dispatch 2024-09-10T18:31:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[18597]: audit 2024-09-10T18:31:06.093365+0000 mon.a (mon.0) 493 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]': finished 2024-09-10T18:31:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[18597]: cluster 2024-09-10T18:31:06.093620+0000 mon.a (mon.0) 494 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:31:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[18597]: audit 2024-09-10T18:31:06.093792+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[31607]: cluster 2024-09-10T18:31:05.233499+0000 mgr.y (mgr.14150) 255 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[31607]: audit 2024-09-10T18:31:06.068349+0000 mon.b (mon.1) 14 : audit [INF] from='client.? 172.21.15.151:0/2976399911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]: dispatch 2024-09-10T18:31:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[31607]: audit 2024-09-10T18:31:06.079478+0000 mon.a (mon.0) 492 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]: dispatch 2024-09-10T18:31:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[31607]: audit 2024-09-10T18:31:06.093365+0000 mon.a (mon.0) 493 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "115d5a71-f0c1-4d54-8cb7-7e190a5405a3"}]': finished 2024-09-10T18:31:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[31607]: cluster 2024-09-10T18:31:06.093620+0000 mon.a (mon.0) 494 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:31:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:06 smithi100 bash[31607]: audit 2024-09-10T18:31:06.093792+0000 mon.a (mon.0) 495 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:07.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:07 smithi151 bash[22565]: audit 2024-09-10T18:31:06.764991+0000 mon.b (mon.1) 15 : audit [DBG] from='client.? 172.21.15.151:0/1185785765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:31:07.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:07 smithi100 bash[18597]: audit 2024-09-10T18:31:06.764991+0000 mon.b (mon.1) 15 : audit [DBG] from='client.? 172.21.15.151:0/1185785765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:31:07.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:07 smithi100 bash[31607]: audit 2024-09-10T18:31:06.764991+0000 mon.b (mon.1) 15 : audit [DBG] from='client.? 172.21.15.151:0/1185785765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:31:08.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:08 smithi151 bash[22565]: cluster 2024-09-10T18:31:07.234031+0000 mgr.y (mgr.14150) 256 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:08 smithi100 bash[18597]: cluster 2024-09-10T18:31:07.234031+0000 mgr.y (mgr.14150) 256 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:08.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:08 smithi100 bash[31607]: cluster 2024-09-10T18:31:07.234031+0000 mgr.y (mgr.14150) 256 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:10.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:10 smithi151 bash[22565]: cluster 2024-09-10T18:31:09.234619+0000 mgr.y (mgr.14150) 257 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:10 smithi100 bash[18597]: cluster 2024-09-10T18:31:09.234619+0000 mgr.y (mgr.14150) 257 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:10.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:10 smithi100 bash[31607]: cluster 2024-09-10T18:31:09.234619+0000 mgr.y (mgr.14150) 257 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:12.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:12 smithi151 bash[22565]: cluster 2024-09-10T18:31:11.235177+0000 mgr.y (mgr.14150) 258 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:12 smithi100 bash[18597]: cluster 2024-09-10T18:31:11.235177+0000 mgr.y (mgr.14150) 258 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:12 smithi100 bash[31607]: cluster 2024-09-10T18:31:11.235177+0000 mgr.y (mgr.14150) 258 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:14.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:14 smithi151 bash[22565]: cluster 2024-09-10T18:31:13.235722+0000 mgr.y (mgr.14150) 259 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:14.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:14 smithi100 bash[18597]: cluster 2024-09-10T18:31:13.235722+0000 mgr.y (mgr.14150) 259 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:14.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:14 smithi100 bash[31607]: cluster 2024-09-10T18:31:13.235722+0000 mgr.y (mgr.14150) 259 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:16.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:16 smithi151 bash[22565]: cluster 2024-09-10T18:31:15.236252+0000 mgr.y (mgr.14150) 260 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:16 smithi100 bash[18597]: cluster 2024-09-10T18:31:15.236252+0000 mgr.y (mgr.14150) 260 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:16.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:16 smithi100 bash[31607]: cluster 2024-09-10T18:31:15.236252+0000 mgr.y (mgr.14150) 260 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:18.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:18 smithi151 bash[22565]: cluster 2024-09-10T18:31:17.236795+0000 mgr.y (mgr.14150) 261 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:18.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:18 smithi100 bash[18597]: cluster 2024-09-10T18:31:17.236795+0000 mgr.y (mgr.14150) 261 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:18 smithi100 bash[31607]: cluster 2024-09-10T18:31:17.236795+0000 mgr.y (mgr.14150) 261 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:19.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:19 smithi151 bash[22565]: audit 2024-09-10T18:31:18.738877+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:31:19.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:19 smithi151 bash[22565]: audit 2024-09-10T18:31:18.740539+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:19.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:19 smithi151 bash[22565]: cephadm 2024-09-10T18:31:18.742047+0000 mgr.y (mgr.14150) 262 : cephadm [INF] Deploying daemon osd.5 on smithi151 2024-09-10T18:31:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:19 smithi100 bash[18597]: audit 2024-09-10T18:31:18.738877+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:31:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:19 smithi100 bash[18597]: audit 2024-09-10T18:31:18.740539+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:19 smithi100 bash[18597]: cephadm 2024-09-10T18:31:18.742047+0000 mgr.y (mgr.14150) 262 : cephadm [INF] Deploying daemon osd.5 on smithi151 2024-09-10T18:31:19.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:19 smithi100 bash[31607]: audit 2024-09-10T18:31:18.738877+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:31:19.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:19 smithi100 bash[31607]: audit 2024-09-10T18:31:18.740539+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:19 smithi100 bash[31607]: cephadm 2024-09-10T18:31:18.742047+0000 mgr.y (mgr.14150) 262 : cephadm [INF] Deploying daemon osd.5 on smithi151 2024-09-10T18:31:20.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:20 smithi151 bash[22565]: cluster 2024-09-10T18:31:19.237428+0000 mgr.y (mgr.14150) 263 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:20.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:20 smithi100 bash[18597]: cluster 2024-09-10T18:31:19.237428+0000 mgr.y (mgr.14150) 263 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:20.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:20 smithi100 bash[31607]: cluster 2024-09-10T18:31:19.237428+0000 mgr.y (mgr.14150) 263 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:22.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:22 smithi151 bash[22565]: cluster 2024-09-10T18:31:21.238018+0000 mgr.y (mgr.14150) 264 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:22 smithi100 bash[18597]: cluster 2024-09-10T18:31:21.238018+0000 mgr.y (mgr.14150) 264 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:22.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:22 smithi100 bash[31607]: cluster 2024-09-10T18:31:21.238018+0000 mgr.y (mgr.14150) 264 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:24.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:24 smithi151 bash[22565]: cluster 2024-09-10T18:31:23.238606+0000 mgr.y (mgr.14150) 265 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:24.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:24 smithi151 bash[22565]: audit 2024-09-10T18:31:23.543828+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:31:24.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:24 smithi151 bash[22565]: audit 2024-09-10T18:31:23.552878+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:24.666 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:24 smithi151 bash[22565]: audit 2024-09-10T18:31:23.560968+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[18597]: cluster 2024-09-10T18:31:23.238606+0000 mgr.y (mgr.14150) 265 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[18597]: audit 2024-09-10T18:31:23.543828+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:31:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[18597]: audit 2024-09-10T18:31:23.552878+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[18597]: audit 2024-09-10T18:31:23.560968+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[31607]: cluster 2024-09-10T18:31:23.238606+0000 mgr.y (mgr.14150) 265 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[31607]: audit 2024-09-10T18:31:23.543828+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:31:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[31607]: audit 2024-09-10T18:31:23.552878+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:24 smithi100 bash[31607]: audit 2024-09-10T18:31:23.560968+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:26.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:26 smithi151 bash[22565]: cluster 2024-09-10T18:31:25.239178+0000 mgr.y (mgr.14150) 266 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:26 smithi100 bash[18597]: cluster 2024-09-10T18:31:25.239178+0000 mgr.y (mgr.14150) 266 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:26.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:26 smithi100 bash[31607]: cluster 2024-09-10T18:31:25.239178+0000 mgr.y (mgr.14150) 266 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:28.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:28 smithi151 bash[22565]: cluster 2024-09-10T18:31:27.239720+0000 mgr.y (mgr.14150) 267 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:28 smithi100 bash[18597]: cluster 2024-09-10T18:31:27.239720+0000 mgr.y (mgr.14150) 267 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:28 smithi100 bash[31607]: cluster 2024-09-10T18:31:27.239720+0000 mgr.y (mgr.14150) 267 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: cluster 2024-09-10T18:31:29.240273+0000 mgr.y (mgr.14150) 268 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: audit 2024-09-10T18:31:29.714927+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: audit 2024-09-10T18:31:29.724159+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: audit 2024-09-10T18:31:29.725977+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: audit 2024-09-10T18:31:29.727447+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: audit 2024-09-10T18:31:29.736203+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: audit 2024-09-10T18:31:30.181613+0000 mon.b (mon.1) 16 : audit [INF] from='osd.5 v2:172.21.15.151:6804/3330203218' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:31:30.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:30 smithi151 bash[22565]: audit 2024-09-10T18:31:30.193171+0000 mon.a (mon.0) 506 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:31:31.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: cluster 2024-09-10T18:31:29.240273+0000 mgr.y (mgr.14150) 268 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:31.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: audit 2024-09-10T18:31:29.714927+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:31.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: audit 2024-09-10T18:31:29.724159+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:31.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: audit 2024-09-10T18:31:29.725977+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:31.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: audit 2024-09-10T18:31:29.727447+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: audit 2024-09-10T18:31:29.736203+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: audit 2024-09-10T18:31:30.181613+0000 mon.b (mon.1) 16 : audit [INF] from='osd.5 v2:172.21.15.151:6804/3330203218' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[18597]: audit 2024-09-10T18:31:30.193171+0000 mon.a (mon.0) 506 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: cluster 2024-09-10T18:31:29.240273+0000 mgr.y (mgr.14150) 268 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: audit 2024-09-10T18:31:29.714927+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: audit 2024-09-10T18:31:29.724159+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: audit 2024-09-10T18:31:29.725977+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: audit 2024-09-10T18:31:29.727447+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: audit 2024-09-10T18:31:29.736203+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: audit 2024-09-10T18:31:30.181613+0000 mon.b (mon.1) 16 : audit [INF] from='osd.5 v2:172.21.15.151:6804/3330203218' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:31:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:30 smithi100 bash[31607]: audit 2024-09-10T18:31:30.193171+0000 mon.a (mon.0) 506 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:31:31.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:31 smithi151 bash[22565]: audit 2024-09-10T18:31:30.730911+0000 mon.b (mon.1) 17 : audit [INF] from='osd.5 v2:172.21.15.151:6804/3330203218' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:31:31.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:31 smithi151 bash[22565]: audit 2024-09-10T18:31:30.740312+0000 mon.a (mon.0) 507 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:31:31.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:31 smithi151 bash[22565]: cluster 2024-09-10T18:31:30.740460+0000 mon.a (mon.0) 508 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:31:31.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:31 smithi151 bash[22565]: audit 2024-09-10T18:31:30.740745+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:31.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:31 smithi151 bash[22565]: audit 2024-09-10T18:31:30.742633+0000 mon.a (mon.0) 510 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:31:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[18597]: audit 2024-09-10T18:31:30.730911+0000 mon.b (mon.1) 17 : audit [INF] from='osd.5 v2:172.21.15.151:6804/3330203218' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:31:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[18597]: audit 2024-09-10T18:31:30.740312+0000 mon.a (mon.0) 507 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:31:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[18597]: cluster 2024-09-10T18:31:30.740460+0000 mon.a (mon.0) 508 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:31:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[18597]: audit 2024-09-10T18:31:30.740745+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[18597]: audit 2024-09-10T18:31:30.742633+0000 mon.a (mon.0) 510 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:31:32.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[31607]: audit 2024-09-10T18:31:30.730911+0000 mon.b (mon.1) 17 : audit [INF] from='osd.5 v2:172.21.15.151:6804/3330203218' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:31:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[31607]: audit 2024-09-10T18:31:30.740312+0000 mon.a (mon.0) 507 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:31:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[31607]: cluster 2024-09-10T18:31:30.740460+0000 mon.a (mon.0) 508 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:31:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[31607]: audit 2024-09-10T18:31:30.740745+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:31 smithi100 bash[31607]: audit 2024-09-10T18:31:30.742633+0000 mon.a (mon.0) 510 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:31:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[18597]: cluster 2024-09-10T18:31:31.240825+0000 mgr.y (mgr.14150) 269 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[18597]: audit 2024-09-10T18:31:31.743624+0000 mon.a (mon.0) 511 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:31:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[18597]: cluster 2024-09-10T18:31:31.743910+0000 mon.a (mon.0) 512 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:31:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[18597]: audit 2024-09-10T18:31:31.744636+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[18597]: audit 2024-09-10T18:31:31.752758+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[31607]: cluster 2024-09-10T18:31:31.240825+0000 mgr.y (mgr.14150) 269 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[31607]: audit 2024-09-10T18:31:31.743624+0000 mon.a (mon.0) 511 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:31:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[31607]: cluster 2024-09-10T18:31:31.743910+0000 mon.a (mon.0) 512 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:31:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[31607]: audit 2024-09-10T18:31:31.744636+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:32 smithi100 bash[31607]: audit 2024-09-10T18:31:31.752758+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:33.163 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:32 smithi151 bash[22565]: cluster 2024-09-10T18:31:31.240825+0000 mgr.y (mgr.14150) 269 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:33.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:32 smithi151 bash[22565]: audit 2024-09-10T18:31:31.743624+0000 mon.a (mon.0) 511 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:31:33.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:32 smithi151 bash[22565]: cluster 2024-09-10T18:31:31.743910+0000 mon.a (mon.0) 512 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:31:33.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:32 smithi151 bash[22565]: audit 2024-09-10T18:31:31.744636+0000 mon.a (mon.0) 513 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:33.164 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:32 smithi151 bash[22565]: audit 2024-09-10T18:31:31.752758+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.864+0000 7f99997fa640 1 -- 172.21.15.151:0/661076741 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f999c05db50 con 0x7f998806edd0 2024-09-10T18:31:33.878 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 5 on host 'smithi151' 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 >> v2:172.21.15.100:6800/3986579393 conn(0x7f998806edd0 msgr2=0x7f9988071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 --2- 172.21.15.151:0/661076741 >> v2:172.21.15.100:6800/3986579393 conn(0x7f998806edd0 0x7f9988071280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9984004eb0 tx=0x7f998400b040 comp rx=0 tx=0).stop 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 msgr2=0x7f999c10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c10ee10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f998c004750 tx=0x7f998c004d10 comp rx=0 tx=0).stop 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 shutdown_connections 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 --2- 172.21.15.151:0/661076741 >> v2:172.21.15.100:6800/3986579393 conn(0x7f998806edd0 0x7f9988071280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f999c100500 0x7f999c116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f999c0ffb50 0x7f999c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:33.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 --2- 172.21.15.151:0/661076741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c0fe880 0x7f999c10ee10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:33.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 >> 172.21.15.151:0/661076741 conn(0x7f999c0fa060 msgr2=0x7f999c0fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:31:33.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 shutdown_connections 2024-09-10T18:31:33.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:33.868+0000 7f99a1a47640 1 -- 172.21.15.151:0/661076741 wait complete. 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[31607]: cluster 2024-09-10T18:31:31.151053+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[31607]: cluster 2024-09-10T18:31:31.151153+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[31607]: cluster 2024-09-10T18:31:32.755386+0000 mon.a (mon.0) 515 : cluster [INF] osd.5 v2:172.21.15.151:6804/3330203218 boot 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[31607]: cluster 2024-09-10T18:31:32.755533+0000 mon.a (mon.0) 516 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[31607]: audit 2024-09-10T18:31:32.756579+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[18597]: cluster 2024-09-10T18:31:31.151053+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[18597]: cluster 2024-09-10T18:31:31.151153+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:31:34.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[18597]: cluster 2024-09-10T18:31:32.755386+0000 mon.a (mon.0) 515 : cluster [INF] osd.5 v2:172.21.15.151:6804/3330203218 boot 2024-09-10T18:31:34.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[18597]: cluster 2024-09-10T18:31:32.755533+0000 mon.a (mon.0) 516 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:31:34.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:33 smithi100 bash[18597]: audit 2024-09-10T18:31:32.756579+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:34.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:33 smithi151 bash[22565]: cluster 2024-09-10T18:31:31.151053+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:31:34.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:33 smithi151 bash[22565]: cluster 2024-09-10T18:31:31.151153+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:31:34.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:33 smithi151 bash[22565]: cluster 2024-09-10T18:31:32.755386+0000 mon.a (mon.0) 515 : cluster [INF] osd.5 v2:172.21.15.151:6804/3330203218 boot 2024-09-10T18:31:34.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:33 smithi151 bash[22565]: cluster 2024-09-10T18:31:32.755533+0000 mon.a (mon.0) 516 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:31:34.119 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:33 smithi151 bash[22565]: audit 2024-09-10T18:31:32.756579+0000 mon.a (mon.0) 517 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:31:34.632 DEBUG:teuthology.orchestra.run.smithi151:osd.5> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:31:34.635 INFO:tasks.cephadm:Deploying osd.6 on smithi151 with /dev/vg_nvme/lv_2... 2024-09-10T18:31:34.635 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:31:34.857 INFO:journalctl@ceph.osd.5.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:31:34.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:34 smithi151 bash[22565]: cluster 2024-09-10T18:31:33.241327+0000 mgr.y (mgr.14150) 270 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:34 smithi151 bash[22565]: cluster 2024-09-10T18:31:33.758449+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:31:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:34 smithi151 bash[22565]: audit 2024-09-10T18:31:33.859351+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:31:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:34 smithi151 bash[22565]: audit 2024-09-10T18:31:33.866387+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:34 smithi151 bash[22565]: audit 2024-09-10T18:31:33.874325+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[18597]: cluster 2024-09-10T18:31:33.241327+0000 mgr.y (mgr.14150) 270 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[18597]: cluster 2024-09-10T18:31:33.758449+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:31:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[18597]: audit 2024-09-10T18:31:33.859351+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:31:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[18597]: audit 2024-09-10T18:31:33.866387+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[18597]: audit 2024-09-10T18:31:33.874325+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[31607]: cluster 2024-09-10T18:31:33.241327+0000 mgr.y (mgr.14150) 270 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:31:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[31607]: cluster 2024-09-10T18:31:33.758449+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:31:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[31607]: audit 2024-09-10T18:31:33.859351+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:31:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[31607]: audit 2024-09-10T18:31:33.866387+0000 mon.a (mon.0) 520 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:34 smithi100 bash[31607]: audit 2024-09-10T18:31:33.874325+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:36.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:35 smithi100 bash[18597]: cluster 2024-09-10T18:31:34.776548+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:31:36.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:35 smithi100 bash[31607]: cluster 2024-09-10T18:31:34.776548+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:31:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:35 smithi151 bash[22565]: cluster 2024-09-10T18:31:34.776548+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:31:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:36 smithi100 bash[18597]: cluster 2024-09-10T18:31:35.241850+0000 mgr.y (mgr.14150) 271 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:37.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:36 smithi100 bash[31607]: cluster 2024-09-10T18:31:35.241850+0000 mgr.y (mgr.14150) 271 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:37.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:36 smithi151 bash[22565]: cluster 2024-09-10T18:31:35.241850+0000 mgr.y (mgr.14150) 271 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:38.638 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:31:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:38 smithi100 bash[18597]: cluster 2024-09-10T18:31:37.242406+0000 mgr.y (mgr.14150) 272 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:39.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:38 smithi100 bash[31607]: cluster 2024-09-10T18:31:37.242406+0000 mgr.y (mgr.14150) 272 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:39.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:38 smithi151 bash[22565]: cluster 2024-09-10T18:31:37.242406+0000 mgr.y (mgr.14150) 272 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:40.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:40 smithi151 bash[22565]: cluster 2024-09-10T18:31:39.243011+0000 mgr.y (mgr.14150) 273 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:41.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:40 smithi100 bash[18597]: cluster 2024-09-10T18:31:39.243011+0000 mgr.y (mgr.14150) 273 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:40 smithi100 bash[31607]: cluster 2024-09-10T18:31:39.243011+0000 mgr.y (mgr.14150) 273 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:43.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:42 smithi151 bash[22565]: cluster 2024-09-10T18:31:41.243571+0000 mgr.y (mgr.14150) 274 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:42 smithi100 bash[18597]: cluster 2024-09-10T18:31:41.243571+0000 mgr.y (mgr.14150) 274 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:43.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:42 smithi100 bash[31607]: cluster 2024-09-10T18:31:41.243571+0000 mgr.y (mgr.14150) 274 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:44.021 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:31:44.218 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:31:44.244 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-09-10T18:31:44.244 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-09-10T18:31:44.244 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0223415 s, 469 MB/s 2024-09-10T18:31:44.244 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-09-10T18:31:45.054 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:44 smithi151 bash[22565]: cluster 2024-09-10T18:31:43.244120+0000 mgr.y (mgr.14150) 275 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:45.055 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-09-10T18:31:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:44 smithi100 bash[18597]: cluster 2024-09-10T18:31:43.244120+0000 mgr.y (mgr.14150) 275 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:45.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:44 smithi100 bash[31607]: cluster 2024-09-10T18:31:43.244120+0000 mgr.y (mgr.14150) 275 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: cluster 2024-09-10T18:31:45.244725+0000 mgr.y (mgr.14150) 276 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: cephadm 2024-09-10T18:31:45.804796+0000 mgr.y (mgr.14150) 277 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.813395+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.819720+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.820991+0000 mon.a (mon.0) 525 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.822221+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: cephadm 2024-09-10T18:31:45.823113+0000 mgr.y (mgr.14150) 278 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 8635M 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.831720+0000 mon.a (mon.0) 527 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.833541+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:47.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.835065+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:31:47.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:46 smithi151 bash[22565]: audit 2024-09-10T18:31:45.843039+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: cluster 2024-09-10T18:31:45.244725+0000 mgr.y (mgr.14150) 276 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: cephadm 2024-09-10T18:31:45.804796+0000 mgr.y (mgr.14150) 277 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:31:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.813395+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.819720+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.820991+0000 mon.a (mon.0) 525 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.822221+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: cephadm 2024-09-10T18:31:45.823113+0000 mgr.y (mgr.14150) 278 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 8635M 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.831720+0000 mon.a (mon.0) 527 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.833541+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.835065+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[18597]: audit 2024-09-10T18:31:45.843039+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: cluster 2024-09-10T18:31:45.244725+0000 mgr.y (mgr.14150) 276 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: cephadm 2024-09-10T18:31:45.804796+0000 mgr.y (mgr.14150) 277 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.813395+0000 mon.a (mon.0) 523 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.819720+0000 mon.a (mon.0) 524 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.820991+0000 mon.a (mon.0) 525 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:31:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.822221+0000 mon.a (mon.0) 526 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:31:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: cephadm 2024-09-10T18:31:45.823113+0000 mgr.y (mgr.14150) 278 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 8635M 2024-09-10T18:31:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.831720+0000 mon.a (mon.0) 527 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.833541+0000 mon.a (mon.0) 528 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.835065+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:31:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:46 smithi100 bash[31607]: audit 2024-09-10T18:31:45.843039+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:31:49.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:48 smithi151 bash[22565]: cluster 2024-09-10T18:31:47.245299+0000 mgr.y (mgr.14150) 279 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:49.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:48 smithi100 bash[18597]: cluster 2024-09-10T18:31:47.245299+0000 mgr.y (mgr.14150) 279 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:49.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:48 smithi100 bash[31607]: cluster 2024-09-10T18:31:47.245299+0000 mgr.y (mgr.14150) 279 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:49.793 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:31:51.093 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:50 smithi151 bash[22565]: cluster 2024-09-10T18:31:49.245908+0000 mgr.y (mgr.14150) 280 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:51.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:50 smithi100 bash[18597]: cluster 2024-09-10T18:31:49.245908+0000 mgr.y (mgr.14150) 280 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:51.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:50 smithi100 bash[31607]: cluster 2024-09-10T18:31:49.245908+0000 mgr.y (mgr.14150) 280 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 -- 172.21.15.151:0/3704021317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 msgr2=0x7f87f80ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:31:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/3704021317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f80ff3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87ec004950 tx=0x7f87ec009da0 comp rx=0 tx=0).stop 2024-09-10T18:31:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 -- 172.21.15.151:0/3704021317 shutdown_connections 2024-09-10T18:31:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/3704021317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f8102440 0x7f87f81048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/3704021317 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87f80ff930 0x7f87f8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/3704021317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f80ff3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:53.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 -- 172.21.15.151:0/3704021317 >> 172.21.15.151:0/3704021317 conn(0x7f87f80f6ec0 msgr2=0x7f87f80f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:31:53.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 -- 172.21.15.151:0/3704021317 shutdown_connections 2024-09-10T18:31:53.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:52.996+0000 7f87fd3f9640 1 -- 172.21.15.151:0/3704021317 wait complete. 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 Processor -- start 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 -- start start 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f810ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff930 0x7f87f810d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87f8102440 0x7f87f8114330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8114900 con 0x7f87f80ff930 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8114a70 con 0x7f87f8066c90 2024-09-10T18:31:53.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f8114c30 con 0x7f87f8102440 2024-09-10T18:31:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f810ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:31:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f810ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35152/0 (socket says 172.21.15.151:35152) 2024-09-10T18:31:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 -- 172.21.15.151:0/2347206261 learned_addr learned my addr 172.21.15.151:0/2347206261 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:31:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f67fc640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff930 0x7f87f810d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:31:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f77fe640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87f8102440 0x7f87f8114330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:31:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 -- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87f8102440 msgr2=0x7f87f8114330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:31:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87f8102440 0x7f87f8114330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 -- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff930 msgr2=0x7f87f810d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:31:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff930 0x7f87f810d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 -- 172.21.15.151:0/2347206261 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec005040 con 0x7f87f8066c90 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87f6ffd640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f810ce20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0043d0 tx=0x7f87ec009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87df7fe640 1 -- 172.21.15.151:0/2347206261 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec02ac40 con 0x7f87f8066c90 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87df7fe640 1 -- 172.21.15.151:0/2347206261 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f87ec02ada0 con 0x7f87f8066c90 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f8114eb0 con 0x7f87f8066c90 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f81152a0 con 0x7f87f8066c90 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.000+0000 7f87df7fe640 1 -- 172.21.15.151:0/2347206261 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec033690 con 0x7f87f8066c90 2024-09-10T18:31:53.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.004+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e4005040 con 0x7f87f8066c90 2024-09-10T18:31:53.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.004+0000 7f87df7fe640 1 -- 172.21.15.151:0/2347206261 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7f87ec030020 con 0x7f87f8066c90 2024-09-10T18:31:53.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.004+0000 7f87df7fe640 1 --2- 172.21.15.151:0/2347206261 >> v2:172.21.15.100:6800/3986579393 conn(0x7f87e006edd0 0x7f87e0071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:31:53.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.004+0000 7f87df7fe640 1 -- 172.21.15.151:0/2347206261 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 3493+0+0 (secure 0 0 0) 0x7f87ec0ad0c0 con 0x7f87f8066c90 2024-09-10T18:31:53.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.004+0000 7f87f67fc640 1 --2- 172.21.15.151:0/2347206261 >> v2:172.21.15.100:6800/3986579393 conn(0x7f87e006edd0 0x7f87e0071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:31:53.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.008+0000 7f87f67fc640 1 --2- 172.21.15.151:0/2347206261 >> v2:172.21.15.100:6800/3986579393 conn(0x7f87e006edd0 0x7f87e0071280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87e8004ed0 tx=0x7f87e800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:31:53.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.016+0000 7f87df7fe640 1 -- 172.21.15.151:0/2347206261 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f87ec07aba0 con 0x7f87f8066c90 2024-09-10T18:31:53.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:52 smithi151 bash[22565]: cluster 2024-09-10T18:31:51.246499+0000 mgr.y (mgr.14150) 281 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:53.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:31:53.192+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f87e4000bf0 con 0x7f87e006edd0 2024-09-10T18:31:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:52 smithi100 bash[18597]: cluster 2024-09-10T18:31:51.246499+0000 mgr.y (mgr.14150) 281 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:53.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:52 smithi100 bash[31607]: cluster 2024-09-10T18:31:51.246499+0000 mgr.y (mgr.14150) 281 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:54.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:53 smithi151 bash[22565]: audit 2024-09-10T18:31:53.214017+0000 mon.a (mon.0) 531 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:31:54.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:53 smithi151 bash[22565]: audit 2024-09-10T18:31:53.220112+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:31:54.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:53 smithi151 bash[22565]: audit 2024-09-10T18:31:53.221824+0000 mon.a (mon.0) 533 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:53 smithi100 bash[18597]: audit 2024-09-10T18:31:53.214017+0000 mon.a (mon.0) 531 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:31:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:53 smithi100 bash[18597]: audit 2024-09-10T18:31:53.220112+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:31:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:53 smithi100 bash[18597]: audit 2024-09-10T18:31:53.221824+0000 mon.a (mon.0) 533 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:54.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:53 smithi100 bash[31607]: audit 2024-09-10T18:31:53.214017+0000 mon.a (mon.0) 531 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:31:54.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:53 smithi100 bash[31607]: audit 2024-09-10T18:31:53.220112+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:31:54.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:53 smithi100 bash[31607]: audit 2024-09-10T18:31:53.221824+0000 mon.a (mon.0) 533 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:31:55.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:54 smithi151 bash[22565]: audit 2024-09-10T18:31:53.207793+0000 mgr.y (mgr.14150) 282 : audit [DBG] from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:31:55.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:54 smithi151 bash[22565]: cluster 2024-09-10T18:31:53.247025+0000 mgr.y (mgr.14150) 283 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:54 smithi100 bash[18597]: audit 2024-09-10T18:31:53.207793+0000 mgr.y (mgr.14150) 282 : audit [DBG] from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:31:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:54 smithi100 bash[18597]: cluster 2024-09-10T18:31:53.247025+0000 mgr.y (mgr.14150) 283 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:54 smithi100 bash[31607]: audit 2024-09-10T18:31:53.207793+0000 mgr.y (mgr.14150) 282 : audit [DBG] from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:31:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:54 smithi100 bash[31607]: cluster 2024-09-10T18:31:53.247025+0000 mgr.y (mgr.14150) 283 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:57.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:56 smithi151 bash[22565]: cluster 2024-09-10T18:31:55.247585+0000 mgr.y (mgr.14150) 284 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:57.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:56 smithi100 bash[18597]: cluster 2024-09-10T18:31:55.247585+0000 mgr.y (mgr.14150) 284 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:57.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:56 smithi100 bash[31607]: cluster 2024-09-10T18:31:55.247585+0000 mgr.y (mgr.14150) 284 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:59.142 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:31:58 smithi151 bash[22565]: cluster 2024-09-10T18:31:57.248172+0000 mgr.y (mgr.14150) 285 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:31:58 smithi100 bash[18597]: cluster 2024-09-10T18:31:57.248172+0000 mgr.y (mgr.14150) 285 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:31:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:31:58 smithi100 bash[31607]: cluster 2024-09-10T18:31:57.248172+0000 mgr.y (mgr.14150) 285 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:01.110 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:00 smithi151 bash[22565]: cluster 2024-09-10T18:31:59.248794+0000 mgr.y (mgr.14150) 286 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:00 smithi100 bash[18597]: cluster 2024-09-10T18:31:59.248794+0000 mgr.y (mgr.14150) 286 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:01.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:00 smithi100 bash[31607]: cluster 2024-09-10T18:31:59.248794+0000 mgr.y (mgr.14150) 286 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:03.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:02 smithi151 bash[22565]: cluster 2024-09-10T18:32:01.249402+0000 mgr.y (mgr.14150) 287 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:03.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:02 smithi151 bash[22565]: audit 2024-09-10T18:32:01.887640+0000 mon.b (mon.1) 18 : audit [INF] from='client.? 172.21.15.151:0/2311136900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]: dispatch 2024-09-10T18:32:03.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:02 smithi151 bash[22565]: audit 2024-09-10T18:32:01.899992+0000 mon.a (mon.0) 534 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]: dispatch 2024-09-10T18:32:03.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:02 smithi151 bash[22565]: audit 2024-09-10T18:32:01.908923+0000 mon.a (mon.0) 535 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]': finished 2024-09-10T18:32:03.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:02 smithi151 bash[22565]: cluster 2024-09-10T18:32:01.909084+0000 mon.a (mon.0) 536 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:32:03.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:02 smithi151 bash[22565]: audit 2024-09-10T18:32:01.909330+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:03.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:02 smithi151 bash[22565]: audit 2024-09-10T18:32:02.563097+0000 mon.b (mon.1) 19 : audit [DBG] from='client.? 172.21.15.151:0/3258994712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:32:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[18597]: cluster 2024-09-10T18:32:01.249402+0000 mgr.y (mgr.14150) 287 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[18597]: audit 2024-09-10T18:32:01.887640+0000 mon.b (mon.1) 18 : audit [INF] from='client.? 172.21.15.151:0/2311136900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]: dispatch 2024-09-10T18:32:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[18597]: audit 2024-09-10T18:32:01.899992+0000 mon.a (mon.0) 534 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]: dispatch 2024-09-10T18:32:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[18597]: audit 2024-09-10T18:32:01.908923+0000 mon.a (mon.0) 535 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]': finished 2024-09-10T18:32:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[18597]: cluster 2024-09-10T18:32:01.909084+0000 mon.a (mon.0) 536 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:32:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[18597]: audit 2024-09-10T18:32:01.909330+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[18597]: audit 2024-09-10T18:32:02.563097+0000 mon.b (mon.1) 19 : audit [DBG] from='client.? 172.21.15.151:0/3258994712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:32:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[31607]: cluster 2024-09-10T18:32:01.249402+0000 mgr.y (mgr.14150) 287 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[31607]: audit 2024-09-10T18:32:01.887640+0000 mon.b (mon.1) 18 : audit [INF] from='client.? 172.21.15.151:0/2311136900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]: dispatch 2024-09-10T18:32:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[31607]: audit 2024-09-10T18:32:01.899992+0000 mon.a (mon.0) 534 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]: dispatch 2024-09-10T18:32:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[31607]: audit 2024-09-10T18:32:01.908923+0000 mon.a (mon.0) 535 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dbae406-f3c5-43e0-98bf-9b4d26715d2a"}]': finished 2024-09-10T18:32:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[31607]: cluster 2024-09-10T18:32:01.909084+0000 mon.a (mon.0) 536 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:32:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[31607]: audit 2024-09-10T18:32:01.909330+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:02 smithi100 bash[31607]: audit 2024-09-10T18:32:02.563097+0000 mon.b (mon.1) 19 : audit [DBG] from='client.? 172.21.15.151:0/3258994712' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:32:05.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:04 smithi151 bash[22565]: cluster 2024-09-10T18:32:03.250004+0000 mgr.y (mgr.14150) 288 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:04 smithi100 bash[18597]: cluster 2024-09-10T18:32:03.250004+0000 mgr.y (mgr.14150) 288 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:05.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:04 smithi100 bash[31607]: cluster 2024-09-10T18:32:03.250004+0000 mgr.y (mgr.14150) 288 : cluster [DBG] pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:07.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:06 smithi151 bash[22565]: cluster 2024-09-10T18:32:05.250583+0000 mgr.y (mgr.14150) 289 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:06 smithi100 bash[18597]: cluster 2024-09-10T18:32:05.250583+0000 mgr.y (mgr.14150) 289 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:07.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:06 smithi100 bash[31607]: cluster 2024-09-10T18:32:05.250583+0000 mgr.y (mgr.14150) 289 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:09.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:08 smithi151 bash[22565]: cluster 2024-09-10T18:32:07.251127+0000 mgr.y (mgr.14150) 290 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:09.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:08 smithi100 bash[31607]: cluster 2024-09-10T18:32:07.251127+0000 mgr.y (mgr.14150) 290 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:08 smithi100 bash[18597]: cluster 2024-09-10T18:32:07.251127+0000 mgr.y (mgr.14150) 290 : cluster [DBG] pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:11.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:10 smithi151 bash[22565]: cluster 2024-09-10T18:32:09.251705+0000 mgr.y (mgr.14150) 291 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:11.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:10 smithi100 bash[31607]: cluster 2024-09-10T18:32:09.251705+0000 mgr.y (mgr.14150) 291 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:11.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:10 smithi100 bash[18597]: cluster 2024-09-10T18:32:09.251705+0000 mgr.y (mgr.14150) 291 : cluster [DBG] pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:13.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:12 smithi151 bash[22565]: cluster 2024-09-10T18:32:11.252266+0000 mgr.y (mgr.14150) 292 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:13.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:12 smithi100 bash[18597]: cluster 2024-09-10T18:32:11.252266+0000 mgr.y (mgr.14150) 292 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:13.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:12 smithi100 bash[31607]: cluster 2024-09-10T18:32:11.252266+0000 mgr.y (mgr.14150) 292 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:15.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:14 smithi151 bash[22565]: cluster 2024-09-10T18:32:13.252810+0000 mgr.y (mgr.14150) 293 : cluster [DBG] pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:15.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:14 smithi100 bash[18597]: cluster 2024-09-10T18:32:13.252810+0000 mgr.y (mgr.14150) 293 : cluster [DBG] pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:15.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:14 smithi100 bash[31607]: cluster 2024-09-10T18:32:13.252810+0000 mgr.y (mgr.14150) 293 : cluster [DBG] pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:16.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:15 smithi151 bash[22565]: audit 2024-09-10T18:32:15.072311+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:32:16.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:15 smithi151 bash[22565]: audit 2024-09-10T18:32:15.074434+0000 mon.a (mon.0) 539 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:16.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:15 smithi100 bash[18597]: audit 2024-09-10T18:32:15.072311+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:32:16.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:15 smithi100 bash[18597]: audit 2024-09-10T18:32:15.074434+0000 mon.a (mon.0) 539 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:16.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:15 smithi100 bash[31607]: audit 2024-09-10T18:32:15.072311+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:32:16.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:15 smithi100 bash[31607]: audit 2024-09-10T18:32:15.074434+0000 mon.a (mon.0) 539 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:17.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:16 smithi151 bash[22565]: cephadm 2024-09-10T18:32:15.076161+0000 mgr.y (mgr.14150) 294 : cephadm [INF] Deploying daemon osd.6 on smithi151 2024-09-10T18:32:17.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:16 smithi151 bash[22565]: cluster 2024-09-10T18:32:15.253368+0000 mgr.y (mgr.14150) 295 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:17.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:16 smithi100 bash[18597]: cephadm 2024-09-10T18:32:15.076161+0000 mgr.y (mgr.14150) 294 : cephadm [INF] Deploying daemon osd.6 on smithi151 2024-09-10T18:32:17.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:16 smithi100 bash[18597]: cluster 2024-09-10T18:32:15.253368+0000 mgr.y (mgr.14150) 295 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:17.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:16 smithi100 bash[31607]: cephadm 2024-09-10T18:32:15.076161+0000 mgr.y (mgr.14150) 294 : cephadm [INF] Deploying daemon osd.6 on smithi151 2024-09-10T18:32:17.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:16 smithi100 bash[31607]: cluster 2024-09-10T18:32:15.253368+0000 mgr.y (mgr.14150) 295 : cluster [DBG] pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:19.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:18 smithi151 bash[22565]: cluster 2024-09-10T18:32:17.253939+0000 mgr.y (mgr.14150) 296 : cluster [DBG] pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:19.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:18 smithi100 bash[18597]: cluster 2024-09-10T18:32:17.253939+0000 mgr.y (mgr.14150) 296 : cluster [DBG] pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:19.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:18 smithi100 bash[31607]: cluster 2024-09-10T18:32:17.253939+0000 mgr.y (mgr.14150) 296 : cluster [DBG] pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:21.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:20 smithi151 bash[22565]: cluster 2024-09-10T18:32:19.254584+0000 mgr.y (mgr.14150) 297 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:21.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:20 smithi151 bash[22565]: audit 2024-09-10T18:32:19.937079+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:32:21.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:20 smithi151 bash[22565]: audit 2024-09-10T18:32:19.946585+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:21.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:20 smithi151 bash[22565]: audit 2024-09-10T18:32:19.956784+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[31607]: cluster 2024-09-10T18:32:19.254584+0000 mgr.y (mgr.14150) 297 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[31607]: audit 2024-09-10T18:32:19.937079+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[31607]: audit 2024-09-10T18:32:19.946585+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[31607]: audit 2024-09-10T18:32:19.956784+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[18597]: cluster 2024-09-10T18:32:19.254584+0000 mgr.y (mgr.14150) 297 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[18597]: audit 2024-09-10T18:32:19.937079+0000 mon.a (mon.0) 540 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[18597]: audit 2024-09-10T18:32:19.946585+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:21.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:20 smithi100 bash[18597]: audit 2024-09-10T18:32:19.956784+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:23.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:22 smithi151 bash[22565]: cluster 2024-09-10T18:32:21.255155+0000 mgr.y (mgr.14150) 298 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:23.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:22 smithi100 bash[31607]: cluster 2024-09-10T18:32:21.255155+0000 mgr.y (mgr.14150) 298 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:23.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:22 smithi100 bash[18597]: cluster 2024-09-10T18:32:21.255155+0000 mgr.y (mgr.14150) 298 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:25.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:24 smithi100 bash[18597]: cluster 2024-09-10T18:32:23.255720+0000 mgr.y (mgr.14150) 299 : cluster [DBG] pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:25.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:24 smithi100 bash[31607]: cluster 2024-09-10T18:32:23.255720+0000 mgr.y (mgr.14150) 299 : cluster [DBG] pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:25.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:24 smithi151 bash[22565]: cluster 2024-09-10T18:32:23.255720+0000 mgr.y (mgr.14150) 299 : cluster [DBG] pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:27.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:26 smithi151 bash[22565]: cluster 2024-09-10T18:32:25.256284+0000 mgr.y (mgr.14150) 300 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:27.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:26 smithi151 bash[22565]: audit 2024-09-10T18:32:26.580687+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:27.187 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:26 smithi151 bash[22565]: audit 2024-09-10T18:32:26.587665+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:27.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:26 smithi100 bash[18597]: cluster 2024-09-10T18:32:25.256284+0000 mgr.y (mgr.14150) 300 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:27.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:26 smithi100 bash[18597]: audit 2024-09-10T18:32:26.580687+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:27.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:26 smithi100 bash[18597]: audit 2024-09-10T18:32:26.587665+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:27.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:26 smithi100 bash[31607]: cluster 2024-09-10T18:32:25.256284+0000 mgr.y (mgr.14150) 300 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:27.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:26 smithi100 bash[31607]: audit 2024-09-10T18:32:26.580687+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:27.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:26 smithi100 bash[31607]: audit 2024-09-10T18:32:26.587665+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:28.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[18597]: audit 2024-09-10T18:32:26.963768+0000 mon.a (mon.0) 545 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:28.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[18597]: audit 2024-09-10T18:32:26.965531+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:32:28.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[18597]: audit 2024-09-10T18:32:26.975671+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:28.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[18597]: audit 2024-09-10T18:32:27.097435+0000 mon.b (mon.1) 20 : audit [INF] from='osd.6 v2:172.21.15.151:6808/251427555' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:32:28.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[18597]: audit 2024-09-10T18:32:27.110144+0000 mon.a (mon.0) 548 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:32:28.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[31607]: audit 2024-09-10T18:32:26.963768+0000 mon.a (mon.0) 545 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:28.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[31607]: audit 2024-09-10T18:32:26.965531+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:32:28.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[31607]: audit 2024-09-10T18:32:26.975671+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:28.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[31607]: audit 2024-09-10T18:32:27.097435+0000 mon.b (mon.1) 20 : audit [INF] from='osd.6 v2:172.21.15.151:6808/251427555' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:32:28.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:27 smithi100 bash[31607]: audit 2024-09-10T18:32:27.110144+0000 mon.a (mon.0) 548 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:32:28.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:27 smithi151 bash[22565]: audit 2024-09-10T18:32:26.963768+0000 mon.a (mon.0) 545 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:28.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:27 smithi151 bash[22565]: audit 2024-09-10T18:32:26.965531+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:32:28.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:27 smithi151 bash[22565]: audit 2024-09-10T18:32:26.975671+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:28.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:27 smithi151 bash[22565]: audit 2024-09-10T18:32:27.097435+0000 mon.b (mon.1) 20 : audit [INF] from='osd.6 v2:172.21.15.151:6808/251427555' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:32:28.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:27 smithi151 bash[22565]: audit 2024-09-10T18:32:27.110144+0000 mon.a (mon.0) 548 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:32:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:28 smithi151 bash[22565]: cluster 2024-09-10T18:32:27.256869+0000 mgr.y (mgr.14150) 301 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:28 smithi151 bash[22565]: audit 2024-09-10T18:32:27.974345+0000 mon.b (mon.1) 21 : audit [INF] from='osd.6 v2:172.21.15.151:6808/251427555' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:32:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:28 smithi151 bash[22565]: audit 2024-09-10T18:32:27.985019+0000 mon.a (mon.0) 549 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:32:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:28 smithi151 bash[22565]: cluster 2024-09-10T18:32:27.985167+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:32:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:28 smithi151 bash[22565]: audit 2024-09-10T18:32:27.985429+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:29.203 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:28 smithi151 bash[22565]: audit 2024-09-10T18:32:27.987083+0000 mon.a (mon.0) 552 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:32:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[18597]: cluster 2024-09-10T18:32:27.256869+0000 mgr.y (mgr.14150) 301 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[18597]: audit 2024-09-10T18:32:27.974345+0000 mon.b (mon.1) 21 : audit [INF] from='osd.6 v2:172.21.15.151:6808/251427555' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:32:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[18597]: audit 2024-09-10T18:32:27.985019+0000 mon.a (mon.0) 549 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:32:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[18597]: cluster 2024-09-10T18:32:27.985167+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:32:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[18597]: audit 2024-09-10T18:32:27.985429+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[18597]: audit 2024-09-10T18:32:27.987083+0000 mon.a (mon.0) 552 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:32:29.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[31607]: cluster 2024-09-10T18:32:27.256869+0000 mgr.y (mgr.14150) 301 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:29.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[31607]: audit 2024-09-10T18:32:27.974345+0000 mon.b (mon.1) 21 : audit [INF] from='osd.6 v2:172.21.15.151:6808/251427555' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:32:29.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[31607]: audit 2024-09-10T18:32:27.985019+0000 mon.a (mon.0) 549 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:32:29.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[31607]: cluster 2024-09-10T18:32:27.985167+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:32:29.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[31607]: audit 2024-09-10T18:32:27.985429+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:29.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:28 smithi100 bash[31607]: audit 2024-09-10T18:32:27.987083+0000 mon.a (mon.0) 552 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:32:30.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: audit 2024-09-10T18:32:28.989739+0000 mon.a (mon.0) 553 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:32:30.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: cluster 2024-09-10T18:32:28.989854+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-09-10T18:32:30.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: audit 2024-09-10T18:32:28.990179+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: audit 2024-09-10T18:32:28.998521+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: cluster 2024-09-10T18:32:29.167732+0000 mon.a (mon.0) 557 : cluster [INF] osd.6 v2:172.21.15.151:6808/251427555 boot 2024-09-10T18:32:30.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: cluster 2024-09-10T18:32:29.167850+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:32:30.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: audit 2024-09-10T18:32:29.168151+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:29 smithi151 bash[22565]: cluster 2024-09-10T18:32:29.257532+0000 mgr.y (mgr.14150) 302 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: audit 2024-09-10T18:32:28.989739+0000 mon.a (mon.0) 553 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:32:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: cluster 2024-09-10T18:32:28.989854+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-09-10T18:32:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: audit 2024-09-10T18:32:28.990179+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: audit 2024-09-10T18:32:28.998521+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: cluster 2024-09-10T18:32:29.167732+0000 mon.a (mon.0) 557 : cluster [INF] osd.6 v2:172.21.15.151:6808/251427555 boot 2024-09-10T18:32:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: cluster 2024-09-10T18:32:29.167850+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:32:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: audit 2024-09-10T18:32:29.168151+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[18597]: cluster 2024-09-10T18:32:29.257532+0000 mgr.y (mgr.14150) 302 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: audit 2024-09-10T18:32:28.989739+0000 mon.a (mon.0) 553 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: cluster 2024-09-10T18:32:28.989854+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: audit 2024-09-10T18:32:28.990179+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: audit 2024-09-10T18:32:28.998521+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: cluster 2024-09-10T18:32:29.167732+0000 mon.a (mon.0) 557 : cluster [INF] osd.6 v2:172.21.15.151:6808/251427555 boot 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: cluster 2024-09-10T18:32:29.167850+0000 mon.a (mon.0) 558 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: audit 2024-09-10T18:32:29.168151+0000 mon.a (mon.0) 559 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:32:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:29 smithi100 bash[31607]: cluster 2024-09-10T18:32:29.257532+0000 mgr.y (mgr.14150) 302 : cluster [DBG] pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:32:30.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.443+0000 7f87df7fe640 1 -- 172.21.15.151:0/2347206261 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f87e4000bf0 con 0x7f87e006edd0 2024-09-10T18:32:30.459 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 6 on host 'smithi151' 2024-09-10T18:32:30.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 >> v2:172.21.15.100:6800/3986579393 conn(0x7f87e006edd0 msgr2=0x7f87e0071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/2347206261 >> v2:172.21.15.100:6800/3986579393 conn(0x7f87e006edd0 0x7f87e0071280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87e8004ed0 tx=0x7f87e800b040 comp rx=0 tx=0).stop 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 msgr2=0x7f87f810ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f810ce20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0043d0 tx=0x7f87ec009da0 comp rx=0 tx=0).stop 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 shutdown_connections 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/2347206261 >> v2:172.21.15.100:6800/3986579393 conn(0x7f87e006edd0 0x7f87e0071280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87f8102440 0x7f87f8114330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80ff930 0x7f87f810d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 --2- 172.21.15.151:0/2347206261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8066c90 0x7f87f810ce20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 >> 172.21.15.151:0/2347206261 conn(0x7f87f80f6ec0 msgr2=0x7f87f81028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 shutdown_connections 2024-09-10T18:32:30.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:30.447+0000 7f87fd3f9640 1 -- 172.21.15.151:0/2347206261 wait complete. 2024-09-10T18:32:31.201 DEBUG:teuthology.orchestra.run.smithi151:osd.6> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T18:32:31.203 INFO:tasks.cephadm:Deploying osd.7 on smithi151 with /dev/vg_nvme/lv_1... 2024-09-10T18:32:31.203 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:32:31.259 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:30 smithi151 bash[22565]: cluster 2024-09-10T18:32:28.116447+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:32:31.259 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:30 smithi151 bash[22565]: cluster 2024-09-10T18:32:28.116551+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:32:31.260 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:30 smithi151 bash[22565]: cluster 2024-09-10T18:32:30.180513+0000 mon.a (mon.0) 560 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:32:31.260 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:30 smithi151 bash[22565]: audit 2024-09-10T18:32:30.441621+0000 mon.a (mon.0) 561 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:32:31.260 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:30 smithi151 bash[22565]: audit 2024-09-10T18:32:30.449479+0000 mon.a (mon.0) 562 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:31.260 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:30 smithi151 bash[22565]: audit 2024-09-10T18:32:30.456568+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[18597]: cluster 2024-09-10T18:32:28.116447+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:32:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[18597]: cluster 2024-09-10T18:32:28.116551+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:32:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[18597]: cluster 2024-09-10T18:32:30.180513+0000 mon.a (mon.0) 560 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:32:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[18597]: audit 2024-09-10T18:32:30.441621+0000 mon.a (mon.0) 561 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:32:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[18597]: audit 2024-09-10T18:32:30.449479+0000 mon.a (mon.0) 562 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[18597]: audit 2024-09-10T18:32:30.456568+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[31607]: cluster 2024-09-10T18:32:28.116447+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:32:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[31607]: cluster 2024-09-10T18:32:28.116551+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:32:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[31607]: cluster 2024-09-10T18:32:30.180513+0000 mon.a (mon.0) 560 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:32:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[31607]: audit 2024-09-10T18:32:30.441621+0000 mon.a (mon.0) 561 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:32:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[31607]: audit 2024-09-10T18:32:30.449479+0000 mon.a (mon.0) 562 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:30 smithi100 bash[31607]: audit 2024-09-10T18:32:30.456568+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:31.305 INFO:journalctl@ceph.osd.6.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:32:32.542 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:32 smithi100 bash[18597]: cluster 2024-09-10T18:32:31.180689+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-09-10T18:32:32.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:32 smithi100 bash[18597]: cluster 2024-09-10T18:32:31.258241+0000 mgr.y (mgr.14150) 303 : cluster [DBG] pgmap v281: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:32.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:32 smithi100 bash[31607]: cluster 2024-09-10T18:32:31.180689+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-09-10T18:32:32.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:32 smithi100 bash[31607]: cluster 2024-09-10T18:32:31.258241+0000 mgr.y (mgr.14150) 303 : cluster [DBG] pgmap v281: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:32.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:32 smithi151 bash[22565]: cluster 2024-09-10T18:32:31.180689+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-09-10T18:32:32.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:32 smithi151 bash[22565]: cluster 2024-09-10T18:32:31.258241+0000 mgr.y (mgr.14150) 303 : cluster [DBG] pgmap v281: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:33.542 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:33 smithi100 bash[18597]: cluster 2024-09-10T18:32:32.175504+0000 mon.a (mon.0) 565 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:32:33.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:33 smithi100 bash[31607]: cluster 2024-09-10T18:32:32.175504+0000 mon.a (mon.0) 565 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:32:33.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:33 smithi151 bash[22565]: cluster 2024-09-10T18:32:32.175504+0000 mon.a (mon.0) 565 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:32:34.542 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:34 smithi100 bash[18597]: cluster 2024-09-10T18:32:33.258867+0000 mgr.y (mgr.14150) 304 : cluster [DBG] pgmap v282: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:34.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:34 smithi100 bash[31607]: cluster 2024-09-10T18:32:33.258867+0000 mgr.y (mgr.14150) 304 : cluster [DBG] pgmap v282: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:34.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:34 smithi151 bash[22565]: cluster 2024-09-10T18:32:33.258867+0000 mgr.y (mgr.14150) 304 : cluster [DBG] pgmap v282: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:35.240 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:32:35.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:35 smithi151 bash[22565]: cluster 2024-09-10T18:32:35.310598+0000 mon.a (mon.0) 566 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T18:32:35.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:35 smithi151 bash[22565]: cluster 2024-09-10T18:32:35.310664+0000 mon.a (mon.0) 567 : cluster [INF] Cluster is now healthy 2024-09-10T18:32:35.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:35 smithi100 bash[18597]: cluster 2024-09-10T18:32:35.310598+0000 mon.a (mon.0) 566 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T18:32:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:35 smithi100 bash[18597]: cluster 2024-09-10T18:32:35.310664+0000 mon.a (mon.0) 567 : cluster [INF] Cluster is now healthy 2024-09-10T18:32:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:35 smithi100 bash[31607]: cluster 2024-09-10T18:32:35.310598+0000 mon.a (mon.0) 566 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T18:32:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:35 smithi100 bash[31607]: cluster 2024-09-10T18:32:35.310664+0000 mon.a (mon.0) 567 : cluster [INF] Cluster is now healthy 2024-09-10T18:32:36.698 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:36 smithi151 bash[22565]: cluster 2024-09-10T18:32:35.259467+0000 mgr.y (mgr.14150) 305 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:36.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:36 smithi100 bash[18597]: cluster 2024-09-10T18:32:35.259467+0000 mgr.y (mgr.14150) 305 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:36.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:36 smithi100 bash[31607]: cluster 2024-09-10T18:32:35.259467+0000 mgr.y (mgr.14150) 305 : cluster [DBG] pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:38.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:38 smithi151 bash[22565]: cluster 2024-09-10T18:32:37.260021+0000 mgr.y (mgr.14150) 306 : cluster [DBG] pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:38.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:38 smithi100 bash[18597]: cluster 2024-09-10T18:32:37.260021+0000 mgr.y (mgr.14150) 306 : cluster [DBG] pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:38.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:38 smithi100 bash[31607]: cluster 2024-09-10T18:32:37.260021+0000 mgr.y (mgr.14150) 306 : cluster [DBG] pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:40.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:40 smithi151 bash[22565]: cluster 2024-09-10T18:32:39.260588+0000 mgr.y (mgr.14150) 307 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:32:40.687 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:32:40.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:40 smithi100 bash[18597]: cluster 2024-09-10T18:32:39.260588+0000 mgr.y (mgr.14150) 307 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:32:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:40 smithi100 bash[31607]: cluster 2024-09-10T18:32:39.260588+0000 mgr.y (mgr.14150) 307 : cluster [DBG] pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:32:40.875 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:32:40.901 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-09-10T18:32:40.901 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-09-10T18:32:40.901 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0235969 s, 444 MB/s 2024-09-10T18:32:40.902 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-09-10T18:32:41.668 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-09-10T18:32:42.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:42 smithi151 bash[22565]: cluster 2024-09-10T18:32:41.261263+0000 mgr.y (mgr.14150) 308 : cluster [DBG] pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T18:32:42.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:42 smithi100 bash[18597]: cluster 2024-09-10T18:32:41.261263+0000 mgr.y (mgr.14150) 308 : cluster [DBG] pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T18:32:42.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:42 smithi100 bash[31607]: cluster 2024-09-10T18:32:41.261263+0000 mgr.y (mgr.14150) 308 : cluster [DBG] pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T18:32:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: cephadm 2024-09-10T18:32:42.447303+0000 mgr.y (mgr.14150) 309 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:32:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.456446+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.463331+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.464792+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.466345+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.467547+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: cephadm 2024-09-10T18:32:42.468387+0000 mgr.y (mgr.14150) 310 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 5757M 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.474212+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.476118+0000 mon.a (mon.0) 574 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.477508+0000 mon.a (mon.0) 575 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[18597]: audit 2024-09-10T18:32:42.486477+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: cephadm 2024-09-10T18:32:42.447303+0000 mgr.y (mgr.14150) 309 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.456446+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.463331+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.464792+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.466345+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.467547+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: cephadm 2024-09-10T18:32:42.468387+0000 mgr.y (mgr.14150) 310 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 5757M 2024-09-10T18:32:43.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.474212+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.476118+0000 mon.a (mon.0) 574 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:43.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.477508+0000 mon.a (mon.0) 575 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:32:43.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:43 smithi100 bash[31607]: audit 2024-09-10T18:32:42.486477+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: cephadm 2024-09-10T18:32:42.447303+0000 mgr.y (mgr.14150) 309 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.456446+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.463331+0000 mon.a (mon.0) 569 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.464792+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.466345+0000 mon.a (mon.0) 571 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.467547+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: cephadm 2024-09-10T18:32:42.468387+0000 mgr.y (mgr.14150) 310 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 5757M 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.474212+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.476118+0000 mon.a (mon.0) 574 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:43.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.477508+0000 mon.a (mon.0) 575 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:32:43.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:43 smithi151 bash[22565]: audit 2024-09-10T18:32:42.486477+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:32:44.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:44 smithi100 bash[18597]: cluster 2024-09-10T18:32:43.261895+0000 mgr.y (mgr.14150) 311 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:32:44.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:44 smithi100 bash[31607]: cluster 2024-09-10T18:32:43.261895+0000 mgr.y (mgr.14150) 311 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:32:44.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:44 smithi151 bash[22565]: cluster 2024-09-10T18:32:43.261895+0000 mgr.y (mgr.14150) 311 : cluster [DBG] pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:32:46.402 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:32:46.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:46 smithi100 bash[18597]: cluster 2024-09-10T18:32:45.262479+0000 mgr.y (mgr.14150) 312 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:32:46.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:46 smithi100 bash[31607]: cluster 2024-09-10T18:32:45.262479+0000 mgr.y (mgr.14150) 312 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:32:46.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:46 smithi151 bash[22565]: cluster 2024-09-10T18:32:45.262479+0000 mgr.y (mgr.14150) 312 : cluster [DBG] pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T18:32:48.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:48 smithi100 bash[18597]: cluster 2024-09-10T18:32:47.263040+0000 mgr.y (mgr.14150) 313 : cluster [DBG] pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:48.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:48 smithi100 bash[31607]: cluster 2024-09-10T18:32:47.263040+0000 mgr.y (mgr.14150) 313 : cluster [DBG] pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:48.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:48 smithi151 bash[22565]: cluster 2024-09-10T18:32:47.263040+0000 mgr.y (mgr.14150) 313 : cluster [DBG] pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:49.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- 172.21.15.151:0/2028873519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fdad0 msgr2=0x7fd9cc0fdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:32:49.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/2028873519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fdad0 0x7fd9cc0fdf40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0004950 tx=0x7fd9c0009da0 comp rx=0 tx=0).stop 2024-09-10T18:32:49.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- 172.21.15.151:0/2028873519 shutdown_connections 2024-09-10T18:32:49.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/2028873519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc0fe480 0x7fd9cc104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:49.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/2028873519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fdad0 0x7fd9cc0fdf40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:49.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/2028873519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9cc0fc800 0x7fd9cc0fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:49.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- 172.21.15.151:0/2028873519 >> 172.21.15.151:0/2028873519 conn(0x7fd9cc0f7fc0 msgr2=0x7fd9cc0fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:32:49.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- 172.21.15.151:0/2028873519 shutdown_connections 2024-09-10T18:32:49.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- 172.21.15.151:0/2028873519 wait complete. 2024-09-10T18:32:49.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 Processor -- start 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- start start 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fc800 0x7fd9cc10eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9cc0fdad0 0x7fd9cc10f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc0fe480 0x7fd9cc116490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc116a60 con 0x7fd9cc0fe480 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc116bd0 con 0x7fd9cc0fc800 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d3a14640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc116d90 con 0x7fd9cc0fdad0 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d17b0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fc800 0x7fd9cc10eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d17b0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fc800 0x7fd9cc10eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48552/0 (socket says 172.21.15.151:48552) 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d17b0640 1 -- 172.21.15.151:0/416819459 learned_addr learned my addr 172.21.15.151:0/416819459 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d0faf640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9cc0fdad0 0x7fd9cc10f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.815+0000 7fd9d1fb1640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc0fe480 0x7fd9cc116490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:32:49.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d17b0640 1 -- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9cc0fdad0 msgr2=0x7fd9cc10f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d17b0640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9cc0fdad0 0x7fd9cc10f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d17b0640 1 -- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc0fe480 msgr2=0x7fd9cc116490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d17b0640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc0fe480 0x7fd9cc116490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d17b0640 1 -- 172.21.15.151:0/416819459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0005040 con 0x7fd9cc0fc800 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d17b0640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fc800 0x7fd9cc10eef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00ed60 tx=0x7fd9bc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9c9ffb640 1 -- 172.21.15.151:0/416819459 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc009a40 con 0x7fd9cc0fc800 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc117070 con 0x7fd9cc0fc800 2024-09-10T18:32:49.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc117590 con 0x7fd9cc0fc800 2024-09-10T18:32:49.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9c9ffb640 1 -- 172.21.15.151:0/416819459 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd9bc00f040 con 0x7fd9cc0fc800 2024-09-10T18:32:49.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9c9ffb640 1 -- 172.21.15.151:0/416819459 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc009430 con 0x7fd9cc0fc800 2024-09-10T18:32:49.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.819+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b8005040 con 0x7fd9cc0fc800 2024-09-10T18:32:49.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.823+0000 7fd9c9ffb640 1 -- 172.21.15.151:0/416819459 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7fd9bc009590 con 0x7fd9cc0fc800 2024-09-10T18:32:49.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.823+0000 7fd9c9ffb640 1 --2- 172.21.15.151:0/416819459 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd9b406ee00 0x7fd9b40712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:32:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.823+0000 7fd9c9ffb640 1 -- 172.21.15.151:0/416819459 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 3785+0+0 (secure 0 0 0) 0x7fd9bc08df70 con 0x7fd9cc0fc800 2024-09-10T18:32:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.823+0000 7fd9d0faf640 1 --2- 172.21.15.151:0/416819459 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd9b406ee00 0x7fd9b40712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:32:49.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.823+0000 7fd9d0faf640 1 --2- 172.21.15.151:0/416819459 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd9b406ee00 0x7fd9b40712b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00094b0 tx=0x7fd9c0035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:32:49.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:49.831+0000 7fd9c9ffb640 1 -- 172.21.15.151:0/416819459 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd9bc05b730 con 0x7fd9cc0fc800 2024-09-10T18:32:50.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:32:50.015+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9b8000bf0 con 0x7fd9b406ee00 2024-09-10T18:32:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[18597]: cluster 2024-09-10T18:32:49.263609+0000 mgr.y (mgr.14150) 314 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[18597]: audit 2024-09-10T18:32:50.040284+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:32:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[18597]: audit 2024-09-10T18:32:50.044838+0000 mon.a (mon.0) 578 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:32:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[18597]: audit 2024-09-10T18:32:50.046211+0000 mon.a (mon.0) 579 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:50.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[31607]: cluster 2024-09-10T18:32:49.263609+0000 mgr.y (mgr.14150) 314 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:50.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[31607]: audit 2024-09-10T18:32:50.040284+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:32:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[31607]: audit 2024-09-10T18:32:50.044838+0000 mon.a (mon.0) 578 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:32:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:50 smithi100 bash[31607]: audit 2024-09-10T18:32:50.046211+0000 mon.a (mon.0) 579 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:50.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:50 smithi151 bash[22565]: cluster 2024-09-10T18:32:49.263609+0000 mgr.y (mgr.14150) 314 : cluster [DBG] pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:50.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:50 smithi151 bash[22565]: audit 2024-09-10T18:32:50.040284+0000 mon.a (mon.0) 577 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:32:50.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:50 smithi151 bash[22565]: audit 2024-09-10T18:32:50.044838+0000 mon.a (mon.0) 578 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:32:50.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:50 smithi151 bash[22565]: audit 2024-09-10T18:32:50.046211+0000 mon.a (mon.0) 579 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:32:51.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:51 smithi100 bash[18597]: audit 2024-09-10T18:32:50.034038+0000 mgr.y (mgr.14150) 315 : audit [DBG] from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:32:51.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:51 smithi100 bash[31607]: audit 2024-09-10T18:32:50.034038+0000 mgr.y (mgr.14150) 315 : audit [DBG] from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:32:51.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:51 smithi151 bash[22565]: audit 2024-09-10T18:32:50.034038+0000 mgr.y (mgr.14150) 315 : audit [DBG] from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:32:52.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:52 smithi100 bash[18597]: cluster 2024-09-10T18:32:51.264187+0000 mgr.y (mgr.14150) 316 : cluster [DBG] pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:52 smithi100 bash[31607]: cluster 2024-09-10T18:32:51.264187+0000 mgr.y (mgr.14150) 316 : cluster [DBG] pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:52.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:52 smithi151 bash[22565]: cluster 2024-09-10T18:32:51.264187+0000 mgr.y (mgr.14150) 316 : cluster [DBG] pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:54 smithi100 bash[18597]: cluster 2024-09-10T18:32:53.264788+0000 mgr.y (mgr.14150) 317 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:54 smithi100 bash[31607]: cluster 2024-09-10T18:32:53.264788+0000 mgr.y (mgr.14150) 317 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:54.918 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:54 smithi151 bash[22565]: cluster 2024-09-10T18:32:53.264788+0000 mgr.y (mgr.14150) 317 : cluster [DBG] pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:56.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:56 smithi100 bash[18597]: cluster 2024-09-10T18:32:55.265408+0000 mgr.y (mgr.14150) 318 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:56.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:56 smithi100 bash[31607]: cluster 2024-09-10T18:32:55.265408+0000 mgr.y (mgr.14150) 318 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:56.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:56 smithi151 bash[22565]: cluster 2024-09-10T18:32:55.265408+0000 mgr.y (mgr.14150) 318 : cluster [DBG] pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:58 smithi100 bash[18597]: cluster 2024-09-10T18:32:57.265960+0000 mgr.y (mgr.14150) 319 : cluster [DBG] pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:58.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:58 smithi100 bash[31607]: cluster 2024-09-10T18:32:57.265960+0000 mgr.y (mgr.14150) 319 : cluster [DBG] pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:58.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:58 smithi151 bash[22565]: cluster 2024-09-10T18:32:57.265960+0000 mgr.y (mgr.14150) 319 : cluster [DBG] pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:32:59.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[31607]: audit 2024-09-10T18:32:58.906226+0000 mon.b (mon.1) 22 : audit [INF] from='client.? 172.21.15.151:0/340823605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]: dispatch 2024-09-10T18:32:59.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[31607]: audit 2024-09-10T18:32:58.919403+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]: dispatch 2024-09-10T18:32:59.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[31607]: audit 2024-09-10T18:32:58.928953+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]': finished 2024-09-10T18:32:59.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[31607]: cluster 2024-09-10T18:32:58.929162+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:32:59.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[31607]: audit 2024-09-10T18:32:58.929512+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:32:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[18597]: audit 2024-09-10T18:32:58.906226+0000 mon.b (mon.1) 22 : audit [INF] from='client.? 172.21.15.151:0/340823605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]: dispatch 2024-09-10T18:32:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[18597]: audit 2024-09-10T18:32:58.919403+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]: dispatch 2024-09-10T18:32:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[18597]: audit 2024-09-10T18:32:58.928953+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]': finished 2024-09-10T18:32:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[18597]: cluster 2024-09-10T18:32:58.929162+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:32:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:32:59 smithi100 bash[18597]: audit 2024-09-10T18:32:58.929512+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:32:59.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:59 smithi151 bash[22565]: audit 2024-09-10T18:32:58.906226+0000 mon.b (mon.1) 22 : audit [INF] from='client.? 172.21.15.151:0/340823605' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]: dispatch 2024-09-10T18:32:59.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:59 smithi151 bash[22565]: audit 2024-09-10T18:32:58.919403+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]: dispatch 2024-09-10T18:32:59.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:59 smithi151 bash[22565]: audit 2024-09-10T18:32:58.928953+0000 mon.a (mon.0) 581 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df3a51c0-0655-462f-b528-e4ccd5d130ea"}]': finished 2024-09-10T18:32:59.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:59 smithi151 bash[22565]: cluster 2024-09-10T18:32:58.929162+0000 mon.a (mon.0) 582 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:32:59.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:32:59 smithi151 bash[22565]: audit 2024-09-10T18:32:58.929512+0000 mon.a (mon.0) 583 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:00.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:00 smithi100 bash[31607]: cluster 2024-09-10T18:32:59.266492+0000 mgr.y (mgr.14150) 320 : cluster [DBG] pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:00.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:00 smithi100 bash[31607]: audit 2024-09-10T18:32:59.583989+0000 mon.b (mon.1) 23 : audit [DBG] from='client.? 172.21.15.151:0/891017614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:33:00.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:00 smithi100 bash[18597]: cluster 2024-09-10T18:32:59.266492+0000 mgr.y (mgr.14150) 320 : cluster [DBG] pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:00.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:00 smithi100 bash[18597]: audit 2024-09-10T18:32:59.583989+0000 mon.b (mon.1) 23 : audit [DBG] from='client.? 172.21.15.151:0/891017614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:33:00.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:00 smithi151 bash[22565]: cluster 2024-09-10T18:32:59.266492+0000 mgr.y (mgr.14150) 320 : cluster [DBG] pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:00.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:00 smithi151 bash[22565]: audit 2024-09-10T18:32:59.583989+0000 mon.b (mon.1) 23 : audit [DBG] from='client.? 172.21.15.151:0/891017614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:33:02.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:02 smithi151 bash[22565]: cluster 2024-09-10T18:33:01.267059+0000 mgr.y (mgr.14150) 321 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:02.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:02 smithi100 bash[18597]: cluster 2024-09-10T18:33:01.267059+0000 mgr.y (mgr.14150) 321 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:02.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:02 smithi100 bash[31607]: cluster 2024-09-10T18:33:01.267059+0000 mgr.y (mgr.14150) 321 : cluster [DBG] pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:04.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:04 smithi151 bash[22565]: cluster 2024-09-10T18:33:03.267612+0000 mgr.y (mgr.14150) 322 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:05.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:04 smithi100 bash[18597]: cluster 2024-09-10T18:33:03.267612+0000 mgr.y (mgr.14150) 322 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:04 smithi100 bash[31607]: cluster 2024-09-10T18:33:03.267612+0000 mgr.y (mgr.14150) 322 : cluster [DBG] pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:06.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:06 smithi151 bash[22565]: cluster 2024-09-10T18:33:05.268167+0000 mgr.y (mgr.14150) 323 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:07.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:06 smithi100 bash[18597]: cluster 2024-09-10T18:33:05.268167+0000 mgr.y (mgr.14150) 323 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:07.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:06 smithi100 bash[31607]: cluster 2024-09-10T18:33:05.268167+0000 mgr.y (mgr.14150) 323 : cluster [DBG] pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:08.889 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:08 smithi151 bash[22565]: cluster 2024-09-10T18:33:07.268715+0000 mgr.y (mgr.14150) 324 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:09.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:08 smithi100 bash[18597]: cluster 2024-09-10T18:33:07.268715+0000 mgr.y (mgr.14150) 324 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:09.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:08 smithi100 bash[31607]: cluster 2024-09-10T18:33:07.268715+0000 mgr.y (mgr.14150) 324 : cluster [DBG] pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:10 smithi100 bash[18597]: cluster 2024-09-10T18:33:09.269279+0000 mgr.y (mgr.14150) 325 : cluster [DBG] pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:10.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:10 smithi100 bash[31607]: cluster 2024-09-10T18:33:09.269279+0000 mgr.y (mgr.14150) 325 : cluster [DBG] pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:10.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:10 smithi151 bash[22565]: cluster 2024-09-10T18:33:09.269279+0000 mgr.y (mgr.14150) 325 : cluster [DBG] pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:12.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:12 smithi151 bash[22565]: cluster 2024-09-10T18:33:11.269833+0000 mgr.y (mgr.14150) 326 : cluster [DBG] pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:12.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:12 smithi151 bash[22565]: audit 2024-09-10T18:33:12.136444+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:33:12.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:12 smithi151 bash[22565]: audit 2024-09-10T18:33:12.137962+0000 mon.a (mon.0) 585 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:12 smithi100 bash[18597]: cluster 2024-09-10T18:33:11.269833+0000 mgr.y (mgr.14150) 326 : cluster [DBG] pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:12 smithi100 bash[18597]: audit 2024-09-10T18:33:12.136444+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:33:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:12 smithi100 bash[18597]: audit 2024-09-10T18:33:12.137962+0000 mon.a (mon.0) 585 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:12 smithi100 bash[31607]: cluster 2024-09-10T18:33:11.269833+0000 mgr.y (mgr.14150) 326 : cluster [DBG] pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:12 smithi100 bash[31607]: audit 2024-09-10T18:33:12.136444+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:33:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:12 smithi100 bash[31607]: audit 2024-09-10T18:33:12.137962+0000 mon.a (mon.0) 585 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:13.645 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:13 smithi151 bash[22565]: cephadm 2024-09-10T18:33:12.139389+0000 mgr.y (mgr.14150) 327 : cephadm [INF] Deploying daemon osd.7 on smithi151 2024-09-10T18:33:13.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:13 smithi100 bash[18597]: cephadm 2024-09-10T18:33:12.139389+0000 mgr.y (mgr.14150) 327 : cephadm [INF] Deploying daemon osd.7 on smithi151 2024-09-10T18:33:13.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:13 smithi100 bash[31607]: cephadm 2024-09-10T18:33:12.139389+0000 mgr.y (mgr.14150) 327 : cephadm [INF] Deploying daemon osd.7 on smithi151 2024-09-10T18:33:14.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:14 smithi151 bash[22565]: cluster 2024-09-10T18:33:13.270433+0000 mgr.y (mgr.14150) 328 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:14.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:14 smithi100 bash[18597]: cluster 2024-09-10T18:33:13.270433+0000 mgr.y (mgr.14150) 328 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:14.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:14 smithi100 bash[31607]: cluster 2024-09-10T18:33:13.270433+0000 mgr.y (mgr.14150) 328 : cluster [DBG] pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:16.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:16 smithi151 bash[22565]: cluster 2024-09-10T18:33:15.270987+0000 mgr.y (mgr.14150) 329 : cluster [DBG] pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:16 smithi100 bash[18597]: cluster 2024-09-10T18:33:15.270987+0000 mgr.y (mgr.14150) 329 : cluster [DBG] pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:16.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:16 smithi100 bash[31607]: cluster 2024-09-10T18:33:15.270987+0000 mgr.y (mgr.14150) 329 : cluster [DBG] pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:17.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:17 smithi151 bash[22565]: audit 2024-09-10T18:33:17.102758+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:17.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:17 smithi151 bash[22565]: audit 2024-09-10T18:33:17.109481+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:17.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:17 smithi151 bash[22565]: audit 2024-09-10T18:33:17.117627+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:17 smithi100 bash[18597]: audit 2024-09-10T18:33:17.102758+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:17 smithi100 bash[18597]: audit 2024-09-10T18:33:17.109481+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:17 smithi100 bash[18597]: audit 2024-09-10T18:33:17.117627+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:17.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:17 smithi100 bash[31607]: audit 2024-09-10T18:33:17.102758+0000 mon.a (mon.0) 586 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:17.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:17 smithi100 bash[31607]: audit 2024-09-10T18:33:17.109481+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:17.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:17 smithi100 bash[31607]: audit 2024-09-10T18:33:17.117627+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:18.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:18 smithi151 bash[22565]: cluster 2024-09-10T18:33:17.271380+0000 mgr.y (mgr.14150) 330 : cluster [DBG] pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:18.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:18 smithi100 bash[18597]: cluster 2024-09-10T18:33:17.271380+0000 mgr.y (mgr.14150) 330 : cluster [DBG] pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:18 smithi100 bash[31607]: cluster 2024-09-10T18:33:17.271380+0000 mgr.y (mgr.14150) 330 : cluster [DBG] pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:20.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:20 smithi151 bash[22565]: cluster 2024-09-10T18:33:19.271959+0000 mgr.y (mgr.14150) 331 : cluster [DBG] pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:20.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:20 smithi100 bash[18597]: cluster 2024-09-10T18:33:19.271959+0000 mgr.y (mgr.14150) 331 : cluster [DBG] pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:20.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:20 smithi100 bash[31607]: cluster 2024-09-10T18:33:19.271959+0000 mgr.y (mgr.14150) 331 : cluster [DBG] pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:22.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:22 smithi151 bash[22565]: cluster 2024-09-10T18:33:21.272584+0000 mgr.y (mgr.14150) 332 : cluster [DBG] pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:22.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:22 smithi100 bash[18597]: cluster 2024-09-10T18:33:21.272584+0000 mgr.y (mgr.14150) 332 : cluster [DBG] pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:22.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:22 smithi100 bash[31607]: cluster 2024-09-10T18:33:21.272584+0000 mgr.y (mgr.14150) 332 : cluster [DBG] pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:24.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: cluster 2024-09-10T18:33:23.273124+0000 mgr.y (mgr.14150) 333 : cluster [DBG] pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: audit 2024-09-10T18:33:23.572894+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: audit 2024-09-10T18:33:23.581966+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: audit 2024-09-10T18:33:23.583574+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: audit 2024-09-10T18:33:23.585068+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:33:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: audit 2024-09-10T18:33:23.595096+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: audit 2024-09-10T18:33:24.314257+0000 mon.b (mon.1) 24 : audit [INF] from='osd.7 v2:172.21.15.151:6812/4021034041' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:33:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:24 smithi151 bash[22565]: audit 2024-09-10T18:33:24.327727+0000 mon.a (mon.0) 594 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: cluster 2024-09-10T18:33:23.273124+0000 mgr.y (mgr.14150) 333 : cluster [DBG] pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: audit 2024-09-10T18:33:23.572894+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: audit 2024-09-10T18:33:23.581966+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: audit 2024-09-10T18:33:23.583574+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: audit 2024-09-10T18:33:23.585068+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: audit 2024-09-10T18:33:23.595096+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: audit 2024-09-10T18:33:24.314257+0000 mon.b (mon.1) 24 : audit [INF] from='osd.7 v2:172.21.15.151:6812/4021034041' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:33:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[18597]: audit 2024-09-10T18:33:24.327727+0000 mon.a (mon.0) 594 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: cluster 2024-09-10T18:33:23.273124+0000 mgr.y (mgr.14150) 333 : cluster [DBG] pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: audit 2024-09-10T18:33:23.572894+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: audit 2024-09-10T18:33:23.581966+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: audit 2024-09-10T18:33:23.583574+0000 mon.a (mon.0) 591 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: audit 2024-09-10T18:33:23.585068+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: audit 2024-09-10T18:33:23.595096+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: audit 2024-09-10T18:33:24.314257+0000 mon.b (mon.1) 24 : audit [INF] from='osd.7 v2:172.21.15.151:6812/4021034041' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:33:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:24 smithi100 bash[31607]: audit 2024-09-10T18:33:24.327727+0000 mon.a (mon.0) 594 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:33:25.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:25 smithi151 bash[22565]: audit 2024-09-10T18:33:24.594877+0000 mon.b (mon.1) 25 : audit [INF] from='osd.7 v2:172.21.15.151:6812/4021034041' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:33:25.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:25 smithi151 bash[22565]: audit 2024-09-10T18:33:24.606507+0000 mon.a (mon.0) 595 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:33:25.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:25 smithi151 bash[22565]: cluster 2024-09-10T18:33:24.606603+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:33:25.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:25 smithi151 bash[22565]: audit 2024-09-10T18:33:24.606863+0000 mon.a (mon.0) 597 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:25.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:25 smithi151 bash[22565]: audit 2024-09-10T18:33:24.608274+0000 mon.a (mon.0) 598 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:33:26.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[18597]: audit 2024-09-10T18:33:24.594877+0000 mon.b (mon.1) 25 : audit [INF] from='osd.7 v2:172.21.15.151:6812/4021034041' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:33:26.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[18597]: audit 2024-09-10T18:33:24.606507+0000 mon.a (mon.0) 595 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:33:26.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[18597]: cluster 2024-09-10T18:33:24.606603+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:33:26.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[18597]: audit 2024-09-10T18:33:24.606863+0000 mon.a (mon.0) 597 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:26.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[18597]: audit 2024-09-10T18:33:24.608274+0000 mon.a (mon.0) 598 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:33:26.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[31607]: audit 2024-09-10T18:33:24.594877+0000 mon.b (mon.1) 25 : audit [INF] from='osd.7 v2:172.21.15.151:6812/4021034041' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:33:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[31607]: audit 2024-09-10T18:33:24.606507+0000 mon.a (mon.0) 595 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:33:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[31607]: cluster 2024-09-10T18:33:24.606603+0000 mon.a (mon.0) 596 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:33:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[31607]: audit 2024-09-10T18:33:24.606863+0000 mon.a (mon.0) 597 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:25 smithi100 bash[31607]: audit 2024-09-10T18:33:24.608274+0000 mon.a (mon.0) 598 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-10T18:33:26.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:26 smithi151 bash[22565]: cluster 2024-09-10T18:33:25.273672+0000 mgr.y (mgr.14150) 334 : cluster [DBG] pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:26.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:26 smithi151 bash[22565]: audit 2024-09-10T18:33:25.606265+0000 mon.a (mon.0) 599 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:33:26.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:26 smithi151 bash[22565]: cluster 2024-09-10T18:33:25.606443+0000 mon.a (mon.0) 600 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-09-10T18:33:26.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:26 smithi151 bash[22565]: audit 2024-09-10T18:33:25.607147+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:26.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:26 smithi151 bash[22565]: audit 2024-09-10T18:33:25.615764+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:26.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:26 smithi151 bash[22565]: cluster 2024-09-10T18:33:26.607458+0000 mon.a (mon.0) 603 : cluster [INF] osd.7 v2:172.21.15.151:6812/4021034041 boot 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[18597]: cluster 2024-09-10T18:33:25.273672+0000 mgr.y (mgr.14150) 334 : cluster [DBG] pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[18597]: audit 2024-09-10T18:33:25.606265+0000 mon.a (mon.0) 599 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[18597]: cluster 2024-09-10T18:33:25.606443+0000 mon.a (mon.0) 600 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[18597]: audit 2024-09-10T18:33:25.607147+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[18597]: audit 2024-09-10T18:33:25.615764+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[18597]: cluster 2024-09-10T18:33:26.607458+0000 mon.a (mon.0) 603 : cluster [INF] osd.7 v2:172.21.15.151:6812/4021034041 boot 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[31607]: cluster 2024-09-10T18:33:25.273672+0000 mgr.y (mgr.14150) 334 : cluster [DBG] pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:33:27.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[31607]: audit 2024-09-10T18:33:25.606265+0000 mon.a (mon.0) 599 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-10T18:33:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[31607]: cluster 2024-09-10T18:33:25.606443+0000 mon.a (mon.0) 600 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-09-10T18:33:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[31607]: audit 2024-09-10T18:33:25.607147+0000 mon.a (mon.0) 601 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[31607]: audit 2024-09-10T18:33:25.615764+0000 mon.a (mon.0) 602 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:26 smithi100 bash[31607]: cluster 2024-09-10T18:33:26.607458+0000 mon.a (mon.0) 603 : cluster [INF] osd.7 v2:172.21.15.151:6812/4021034041 boot 2024-09-10T18:33:27.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.723+0000 7fd9c9ffb640 1 -- 172.21.15.151:0/416819459 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd9b8000bf0 con 0x7fd9b406ee00 2024-09-10T18:33:27.739 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 7 on host 'smithi151' 2024-09-10T18:33:27.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd9b406ee00 msgr2=0x7fd9b40712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/416819459 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd9b406ee00 0x7fd9b40712b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00094b0 tx=0x7fd9c0035040 comp rx=0 tx=0).stop 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fc800 msgr2=0x7fd9cc10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fc800 0x7fd9cc10eef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00ed60 tx=0x7fd9bc00b040 comp rx=0 tx=0).stop 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 shutdown_connections 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/416819459 >> v2:172.21.15.100:6800/3986579393 conn(0x7fd9b406ee00 0x7fd9b40712b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9cc0fe480 0x7fd9cc116490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9cc0fdad0 0x7fd9cc10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 --2- 172.21.15.151:0/416819459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9cc0fc800 0x7fd9cc10eef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 >> 172.21.15.151:0/416819459 conn(0x7fd9cc0f7fc0 msgr2=0x7fd9cc0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 shutdown_connections 2024-09-10T18:33:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:33:27.727+0000 7fd9d3a14640 1 -- 172.21.15.151:0/416819459 wait complete. 2024-09-10T18:33:27.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:27 smithi151 bash[22565]: cluster 2024-09-10T18:33:25.354625+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:33:27.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:27 smithi151 bash[22565]: cluster 2024-09-10T18:33:25.354720+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:33:27.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:27 smithi151 bash[22565]: cluster 2024-09-10T18:33:26.607553+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:33:27.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:27 smithi151 bash[22565]: audit 2024-09-10T18:33:26.607825+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[18597]: cluster 2024-09-10T18:33:25.354625+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:33:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[18597]: cluster 2024-09-10T18:33:25.354720+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:33:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[18597]: cluster 2024-09-10T18:33:26.607553+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:33:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[18597]: audit 2024-09-10T18:33:26.607825+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:28.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[31607]: cluster 2024-09-10T18:33:25.354625+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-09-10T18:33:28.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[31607]: cluster 2024-09-10T18:33:25.354720+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-09-10T18:33:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[31607]: cluster 2024-09-10T18:33:26.607553+0000 mon.a (mon.0) 604 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:33:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:27 smithi100 bash[31607]: audit 2024-09-10T18:33:26.607825+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:33:28.494 DEBUG:teuthology.orchestra.run.smithi151:osd.7> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T18:33:28.497 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T18:33:28.497 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:33:28.696 INFO:journalctl@ceph.osd.7.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:33:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[18597]: cluster 2024-09-10T18:33:27.274195+0000 mgr.y (mgr.14150) 335 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[18597]: cluster 2024-09-10T18:33:27.624050+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:33:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[18597]: audit 2024-09-10T18:33:27.718547+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[18597]: audit 2024-09-10T18:33:27.726344+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[18597]: audit 2024-09-10T18:33:27.736755+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[31607]: cluster 2024-09-10T18:33:27.274195+0000 mgr.y (mgr.14150) 335 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[31607]: cluster 2024-09-10T18:33:27.624050+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:33:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[31607]: audit 2024-09-10T18:33:27.718547+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[31607]: audit 2024-09-10T18:33:27.726344+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:28 smithi100 bash[31607]: audit 2024-09-10T18:33:27.736755+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:28.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:28 smithi151 bash[22565]: cluster 2024-09-10T18:33:27.274195+0000 mgr.y (mgr.14150) 335 : cluster [DBG] pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:28.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:28 smithi151 bash[22565]: cluster 2024-09-10T18:33:27.624050+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:33:28.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:28 smithi151 bash[22565]: audit 2024-09-10T18:33:27.718547+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:28.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:28 smithi151 bash[22565]: audit 2024-09-10T18:33:27.726344+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:28.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:28 smithi151 bash[22565]: audit 2024-09-10T18:33:27.736755+0000 mon.a (mon.0) 609 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:29.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:29 smithi151 bash[22565]: cluster 2024-09-10T18:33:28.628317+0000 mon.a (mon.0) 610 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:33:30.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:29 smithi100 bash[18597]: cluster 2024-09-10T18:33:28.628317+0000 mon.a (mon.0) 610 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:33:30.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:29 smithi100 bash[31607]: cluster 2024-09-10T18:33:28.628317+0000 mon.a (mon.0) 610 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:33:30.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:30 smithi151 bash[22565]: cluster 2024-09-10T18:33:29.274751+0000 mgr.y (mgr.14150) 336 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:30 smithi100 bash[31607]: cluster 2024-09-10T18:33:29.274751+0000 mgr.y (mgr.14150) 336 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:31.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:30 smithi100 bash[18597]: cluster 2024-09-10T18:33:29.274751+0000 mgr.y (mgr.14150) 336 : cluster [DBG] pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:32.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:32 smithi151 bash[22565]: cluster 2024-09-10T18:33:31.275286+0000 mgr.y (mgr.14150) 337 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:32 smithi100 bash[31607]: cluster 2024-09-10T18:33:31.275286+0000 mgr.y (mgr.14150) 337 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:32 smithi100 bash[18597]: cluster 2024-09-10T18:33:31.275286+0000 mgr.y (mgr.14150) 337 : cluster [DBG] pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:33.260 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:33:34.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:34 smithi151 bash[22565]: cluster 2024-09-10T18:33:33.275925+0000 mgr.y (mgr.14150) 338 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:34.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:34 smithi100 bash[18597]: cluster 2024-09-10T18:33:33.275925+0000 mgr.y (mgr.14150) 338 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:34 smithi100 bash[31607]: cluster 2024-09-10T18:33:33.275925+0000 mgr.y (mgr.14150) 338 : cluster [DBG] pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- 172.21.15.100:0/3094193822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 msgr2=0x7f5dec104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 --2- 172.21.15.100:0/3094193822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 0x7f5dec104cb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009d00 tx=0x7f5de4004d10 comp rx=0 tx=0).stop 2024-09-10T18:33:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- 172.21.15.100:0/3094193822 shutdown_connections 2024-09-10T18:33:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 --2- 172.21.15.100:0/3094193822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 0x7f5dec104cb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 --2- 172.21.15.100:0/3094193822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dec0fe5e0 0x7f5dec0fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 --2- 172.21.15.100:0/3094193822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fdc90 0x7f5dec0fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- 172.21.15.100:0/3094193822 >> 172.21.15.100:0/3094193822 conn(0x7f5dec0f6490 msgr2=0x7f5dec0f88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- 172.21.15.100:0/3094193822 shutdown_connections 2024-09-10T18:33:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- 172.21.15.100:0/3094193822 wait complete. 2024-09-10T18:33:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 Processor -- start 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- start start 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 0x7f5dec0fd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dec0fdc90 0x7f5dec0fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fe5e0 0x7f5dec0fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0fc780 con 0x7f5dec0fae70 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec116ad0 con 0x7f5dec0fdc90 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2f01640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec116c10 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df1eff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 0x7f5dec0fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2700640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fe5e0 0x7f5dec0fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2700640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fe5e0 0x7f5dec0fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:58366/0 (socket says 172.21.15.100:58366) 2024-09-10T18:33:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df2700640 1 -- 172.21.15.100:0/1110150347 learned_addr learned my addr 172.21.15.100:0/1110150347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.741+0000 7f5df16fe640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dec0fdc90 0x7f5dec0fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2700640 1 -- 172.21.15.100:0/1110150347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dec0fdc90 msgr2=0x7f5dec0fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2700640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dec0fdc90 0x7f5dec0fb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2700640 1 -- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 msgr2=0x7f5dec0fd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2700640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 0x7f5dec0fd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2700640 1 -- 172.21.15.100:0/1110150347 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40098b0 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df1eff640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 0x7f5dec0fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2700640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fe5e0 0x7f5dec0fbff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009cd0 tx=0x7f5de4004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5dea7fc640 1 -- 172.21.15.100:0/1110150347 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4038070 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5dea7fc640 1 -- 172.21.15.100:0/1110150347 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f5de40096e0 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5dea7fc640 1 -- 172.21.15.100:0/1110150347 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de403cd30 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec116e90 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec117220 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec05ed70 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5dea7fc640 1 -- 172.21.15.100:0/1110150347 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7f5de4030070 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.745+0000 7f5dea7fc640 1 --2- 172.21.15.100:0/1110150347 >> v2:172.21.15.100:6800/3986579393 conn(0x7f5dd406edd0 0x7f5dd4071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.749+0000 7f5dea7fc640 1 -- 172.21.15.100:0/1110150347 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7f5de40ae210 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.749+0000 7f5df1eff640 1 --2- 172.21.15.100:0/1110150347 >> v2:172.21.15.100:6800/3986579393 conn(0x7f5dd406edd0 0x7f5dd4071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.749+0000 7f5df1eff640 1 --2- 172.21.15.100:0/1110150347 >> v2:172.21.15.100:6800/3986579393 conn(0x7f5dd406edd0 0x7f5dd4071280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0fd230 tx=0x7f5ddc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:33:36.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.753+0000 7f5dea7fc640 1 -- 172.21.15.100:0/1110150347 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5de407b950 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:36 smithi151 bash[22565]: cluster 2024-09-10T18:33:35.276516+0000 mgr.y (mgr.14150) 339 : cluster [DBG] pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:36.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.929+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5dec107550 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5dea7fc640 1 -- 172.21.15.100:0/1110150347 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v51) v1 ==== 74+0+130 (secure 0 0 0) 0x7f5de4041090 con 0x7f5dec0fe5e0 2024-09-10T18:33:36.932 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 >> v2:172.21.15.100:6800/3986579393 conn(0x7f5dd406edd0 msgr2=0x7f5dd4071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 --2- 172.21.15.100:0/1110150347 >> v2:172.21.15.100:6800/3986579393 conn(0x7f5dd406edd0 0x7f5dd4071280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0fd230 tx=0x7f5ddc00b040 comp rx=0 tx=0).stop 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fe5e0 msgr2=0x7f5dec0fbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fe5e0 0x7f5dec0fbff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009cd0 tx=0x7f5de4004d10 comp rx=0 tx=0).stop 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 shutdown_connections 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 --2- 172.21.15.100:0/1110150347 >> v2:172.21.15.100:6800/3986579393 conn(0x7f5dd406edd0 0x7f5dd4071280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dec0fe5e0 0x7f5dec0fbff0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5dec0fdc90 0x7f5dec0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 --2- 172.21.15.100:0/1110150347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec0fae70 0x7f5dec0fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 >> 172.21.15.100:0/1110150347 conn(0x7f5dec0f6490 msgr2=0x7f5dec0f75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 shutdown_connections 2024-09-10T18:33:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:36.933+0000 7f5df2f01640 1 -- 172.21.15.100:0/1110150347 wait complete. 2024-09-10T18:33:36.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:36 smithi100 bash[18597]: cluster 2024-09-10T18:33:35.276516+0000 mgr.y (mgr.14150) 339 : cluster [DBG] pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:36.957 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:36 smithi100 bash[31607]: cluster 2024-09-10T18:33:35.276516+0000 mgr.y (mgr.14150) 339 : cluster [DBG] pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:37.621 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1725993206,"num_in_osds":8,"osd_in_since":1725993178,"num_remapped_pgs":0} 2024-09-10T18:33:37.631 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:33:37.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:37 smithi151 bash[22565]: audit 2024-09-10T18:33:36.933198+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.100:0/1110150347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:33:38.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:37 smithi100 bash[18597]: audit 2024-09-10T18:33:36.933198+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.100:0/1110150347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:33:38.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:37 smithi100 bash[31607]: audit 2024-09-10T18:33:36.933198+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.100:0/1110150347' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:33:38.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: cluster 2024-09-10T18:33:37.277087+0000 mgr.y (mgr.14150) 340 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:38.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.635412+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:38.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.642408+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:38.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.644016+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:38.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.645296+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:38.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.646474+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:38.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.647733+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:38.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.655419+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:38.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.657170+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:38.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.658558+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:33:38.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:38 smithi151 bash[22565]: audit 2024-09-10T18:33:38.667312+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: cluster 2024-09-10T18:33:37.277087+0000 mgr.y (mgr.14150) 340 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.635412+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.642408+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.644016+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.645296+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.646474+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.647733+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.655419+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.657170+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.658558+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[18597]: audit 2024-09-10T18:33:38.667312+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: cluster 2024-09-10T18:33:37.277087+0000 mgr.y (mgr.14150) 340 : cluster [DBG] pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.635412+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.642408+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.644016+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.645296+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.646474+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.647733+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:33:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.655419+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.657170+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.658558+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:33:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:38 smithi100 bash[31607]: audit 2024-09-10T18:33:38.667312+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:33:40.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:39 smithi100 bash[18597]: cephadm 2024-09-10T18:33:38.627748+0000 mgr.y (mgr.14150) 341 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:33:40.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:39 smithi100 bash[18597]: cephadm 2024-09-10T18:33:38.648717+0000 mgr.y (mgr.14150) 342 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 4317M 2024-09-10T18:33:40.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:39 smithi100 bash[31607]: cephadm 2024-09-10T18:33:38.627748+0000 mgr.y (mgr.14150) 341 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:33:40.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:39 smithi100 bash[31607]: cephadm 2024-09-10T18:33:38.648717+0000 mgr.y (mgr.14150) 342 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 4317M 2024-09-10T18:33:40.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:39 smithi151 bash[22565]: cephadm 2024-09-10T18:33:38.627748+0000 mgr.y (mgr.14150) 341 : cephadm [INF] Detected new or changed devices on smithi151 2024-09-10T18:33:40.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:39 smithi151 bash[22565]: cephadm 2024-09-10T18:33:38.648717+0000 mgr.y (mgr.14150) 342 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 4317M 2024-09-10T18:33:41.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:40 smithi100 bash[18597]: cluster 2024-09-10T18:33:39.277679+0000 mgr.y (mgr.14150) 343 : cluster [DBG] pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:40 smithi100 bash[31607]: cluster 2024-09-10T18:33:39.277679+0000 mgr.y (mgr.14150) 343 : cluster [DBG] pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:41.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:40 smithi151 bash[22565]: cluster 2024-09-10T18:33:39.277679+0000 mgr.y (mgr.14150) 343 : cluster [DBG] pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:42.395 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:33:42.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:42 smithi100 bash[18597]: cluster 2024-09-10T18:33:41.278245+0000 mgr.y (mgr.14150) 344 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:42.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:42 smithi100 bash[31607]: cluster 2024-09-10T18:33:41.278245+0000 mgr.y (mgr.14150) 344 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:43.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:42 smithi151 bash[22565]: cluster 2024-09-10T18:33:41.278245+0000 mgr.y (mgr.14150) 344 : cluster [DBG] pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:45.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:44 smithi100 bash[18597]: cluster 2024-09-10T18:33:43.278820+0000 mgr.y (mgr.14150) 345 : cluster [DBG] pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:45.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:44 smithi100 bash[31607]: cluster 2024-09-10T18:33:43.278820+0000 mgr.y (mgr.14150) 345 : cluster [DBG] pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:45.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:44 smithi151 bash[22565]: cluster 2024-09-10T18:33:43.278820+0000 mgr.y (mgr.14150) 345 : cluster [DBG] pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/421126486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 msgr2=0x7fcbc01095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/421126486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 0x7fcbc01095c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8004950 tx=0x7fcbb8009da0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/421126486 shutdown_connections 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/421126486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 0x7fcbc01095c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/421126486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 0x7fcbc00fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/421126486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbc0102850 0x7fcbc0102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/421126486 >> 172.21.15.100:0/421126486 conn(0x7fcbc00745b0 msgr2=0x7fcbc00749b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/421126486 shutdown_connections 2024-09-10T18:33:45.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.800+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/421126486 wait complete. 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 Processor -- start 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 -- start start 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 0x7fcbc006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 0x7fcbc006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbc0102850 0x7fcbc0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc011b0d0 con 0x7fcbc00fcae0 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc011b240 con 0x7fcbc0102850 2024-09-10T18:33:45.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc011b400 con 0x7fcbc00fd470 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 0x7fcbc006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc513f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 0x7fcbc006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc513f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 0x7fcbc006ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:45456/0 (socket says 172.21.15.100:45456) 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 0x7fcbc006f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52882/0 (socket says 172.21.15.100:52882) 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc513f640 1 -- 172.21.15.100:0/1347935770 learned_addr learned my addr 172.21.15.100:0/1347935770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc6141640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbc0102850 0x7fcbc0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 -- 172.21.15.100:0/1347935770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 msgr2=0x7fcbc006ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 0x7fcbc006ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 -- 172.21.15.100:0/1347935770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbc0102850 msgr2=0x7fcbc0070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbc0102850 0x7fcbc0070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 -- 172.21.15.100:0/1347935770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb8005040 con 0x7fcbc00fcae0 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc6141640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbc0102850 0x7fcbc0070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:33:45.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc5940640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 0x7fcbc006f9e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000db80 tx=0x7fcbb0004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:33:45.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbbe7fc640 1 -- 172.21.15.100:0/1347935770 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0017070 con 0x7fcbc00fcae0 2024-09-10T18:33:45.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbbe7fc640 1 -- 172.21.15.100:0/1347935770 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fcbb000f040 con 0x7fcbc00fcae0 2024-09-10T18:33:45.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc011b6e0 con 0x7fcbc00fcae0 2024-09-10T18:33:45.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc011bc00 con 0x7fcbc00fcae0 2024-09-10T18:33:45.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.804+0000 7fcbbe7fc640 1 -- 172.21.15.100:0/1347935770 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00036a0 con 0x7fcbc00fcae0 2024-09-10T18:33:45.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.808+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac005010 con 0x7fcbc00fcae0 2024-09-10T18:33:45.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.808+0000 7fcbbe7fc640 1 -- 172.21.15.100:0/1347935770 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7fcbb000b8f0 con 0x7fcbc00fcae0 2024-09-10T18:33:45.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.808+0000 7fcbbe7fc640 1 --2- 172.21.15.100:0/1347935770 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcba806edd0 0x7fcba8071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:45.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.808+0000 7fcbc513f640 1 --2- 172.21.15.100:0/1347935770 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcba806edd0 0x7fcba8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:45.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.808+0000 7fcbbe7fc640 1 -- 172.21.15.100:0/1347935770 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7fcbb008d940 con 0x7fcbc00fcae0 2024-09-10T18:33:45.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.808+0000 7fcbc513f640 1 --2- 172.21.15.100:0/1347935770 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcba806edd0 0x7fcba8071280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0071110 tx=0x7fcbb400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:33:45.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.816+0000 7fcbbe7fc640 1 -- 172.21.15.100:0/1347935770 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcbb005b050 con 0x7fcbc00fcae0 2024-09-10T18:33:45.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.992+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbac005590 con 0x7fcbc00fcae0 2024-09-10T18:33:45.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.992+0000 7fcbbe7fc640 1 -- 172.21.15.100:0/1347935770 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+11180 (secure 0 0 0) 0x7fcbb005a9f0 con 0x7fcbc00fcae0 2024-09-10T18:33:45.994 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:33:45.994 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":51,"fsid":"505ebbde-6fa1-11ef-bcea-c7b262605968","created":"2024-09-10T18:23:17.238038+0000","modified":"2024-09-10T18:33:28.618598+0000","last_up_change":"2024-09-10T18:33:26.598092+0000","last_in_change":"2024-09-10T18:32:58.920619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:28:49.721821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"9a69cb5a-7a42-4fb4-861d-55a459379aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":2899184110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2899184110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2899184110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":2899184110}]},"public_addr":"172.21.15.100:6801/2899184110","cluster_addr":"172.21.15.100:6802/2899184110","heartbeat_back_addr":"172.21.15.100:6804/2899184110","heartbeat_front_addr":"172.21.15.100:6803/2899184110","state":["exists","up"]},{"osd":1,"uuid":"b6ad7728-dc92-4291-99cd-18ebfacd6c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6805","nonce":59055423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":59055423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":59055423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":59055423}]},"public_addr":"172.21.15.100:6805/59055423","cluster_addr":"172.21.15.100:6806/59055423","heartbeat_back_addr":"172.21.15.100:6808/59055423","heartbeat_front_addr":"172.21.15.100:6807/59055423","state":["exists","up"]},{"osd":2,"uuid":"ce92944f-0b92-46c8-a6b7-6cdd00f9736a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":370401307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":370401307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":370401307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6811","nonce":370401307}]},"public_addr":"172.21.15.100:6809/370401307","cluster_addr":"172.21.15.100:6810/370401307","heartbeat_back_addr":"172.21.15.100:6812/370401307","heartbeat_front_addr":"172.21.15.100:6811/370401307","state":["exists","up"]},{"osd":3,"uuid":"8340f7e5-8d39-4316-ae1e-809bcf286e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":2754318461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2754318461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2754318461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":2754318461}]},"public_addr":"172.21.15.100:6813/2754318461","cluster_addr":"172.21.15.100:6814/2754318461","heartbeat_back_addr":"172.21.15.100:6816/2754318461","heartbeat_front_addr":"172.21.15.100:6815/2754318461","state":["exists","up"]},{"osd":4,"uuid":"80851ca1-0c33-4ac6-80b5-92bbcaa3dacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2989412742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2989412742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6803","nonce":2989412742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2989412742}]},"public_addr":"172.21.15.151:6800/2989412742","cluster_addr":"172.21.15.151:6801/2989412742","heartbeat_back_addr":"172.21.15.151:6803/2989412742","heartbeat_front_addr":"172.21.15.151:6802/2989412742","state":["exists","up"]},{"osd":5,"uuid":"115d5a71-f0c1-4d54-8cb7-7e190a5405a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3330203218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":3330203218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6807","nonce":3330203218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3330203218}]},"public_addr":"172.21.15.151:6804/3330203218","cluster_addr":"172.21.15.151:6805/3330203218","heartbeat_back_addr":"172.21.15.151:6807/3330203218","heartbeat_front_addr":"172.21.15.151:6806/3330203218","state":["exists","up"]},{"osd":6,"uuid":"3dbae406-f3c5-43e0-98bf-9b4d26715d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":251427555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6809","nonce":251427555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":251427555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":251427555}]},"public_addr":"172.21.15.151:6808/251427555","cluster_addr":"172.21.15.151:6809/251427555","heartbeat_back_addr":"172.21.15.151:6811/251427555","heartbeat_front_addr":"172.21.15.151:6810/251427555","state":["exists","up"]},{"osd":7,"uuid":"df3a51c0-0655-462f-b528-e4ccd5d130ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4021034041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6813","nonce":4021034041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6815","nonce":4021034041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4021034041}]},"public_addr":"172.21.15.151:6812/4021034041","cluster_addr":"172.21.15.151:6813/4021034041","heartbeat_back_addr":"172.21.15.151:6815/4021034041","heartbeat_front_addr":"172.21.15.151:6814/4021034041","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:26:57.052592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:27:51.521280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:28:46.335623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:29:42.412430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:30:36.277604+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:31:31.151158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:32:28.116557+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:33:25.354725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/2285226173":"2024-09-11T18:24:19.124827+0000","172.21.15.100:6800/2893280940":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/41752804":"2024-09-11T18:23:45.175932+0000","172.21.15.100:0/69624316":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/1105054798":"2024-09-11T18:24:19.124827+0000","172.21.15.100:6800/1728522861":"2024-09-11T18:23:45.175932+0000","172.21.15.100:0/4071543765":"2024-09-11T18:23:45.175932+0000","172.21.15.100:0/1686873674":"2024-09-11T18:23:45.175932+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcba806edd0 msgr2=0x7fcba8071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/1347935770 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcba806edd0 0x7fcba8071280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0071110 tx=0x7fcbb400b040 comp rx=0 tx=0).stop 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 msgr2=0x7fcbc006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 0x7fcbc006f9e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000db80 tx=0x7fcbb0004e60 comp rx=0 tx=0).stop 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 shutdown_connections 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/1347935770 >> v2:172.21.15.100:6800/3986579393 conn(0x7fcba806edd0 0x7fcba8071280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbc0102850 0x7fcbc0070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcbc00fd470 0x7fcbc006ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 --2- 172.21.15.100:0/1347935770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbc00fcae0 0x7fcbc006f9e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:45.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 >> 172.21.15.100:0/1347935770 conn(0x7fcbc00745b0 msgr2=0x7fcbc00fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:45.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 shutdown_connections 2024-09-10T18:33:45.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:45.996+0000 7fcbc7ba4640 1 -- 172.21.15.100:0/1347935770 wait complete. 2024-09-10T18:33:46.759 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:28:49.721821+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T18:33:46.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T18:33:47.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:46 smithi100 bash[18597]: cluster 2024-09-10T18:33:45.279382+0000 mgr.y (mgr.14150) 346 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:47.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:46 smithi100 bash[18597]: audit 2024-09-10T18:33:45.994062+0000 mon.a (mon.0) 621 : audit [DBG] from='client.? 172.21.15.100:0/1347935770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:33:47.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:46 smithi100 bash[31607]: cluster 2024-09-10T18:33:45.279382+0000 mgr.y (mgr.14150) 346 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:47.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:46 smithi100 bash[31607]: audit 2024-09-10T18:33:45.994062+0000 mon.a (mon.0) 621 : audit [DBG] from='client.? 172.21.15.100:0/1347935770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:33:47.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:46 smithi151 bash[22565]: cluster 2024-09-10T18:33:45.279382+0000 mgr.y (mgr.14150) 346 : cluster [DBG] pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:47.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:46 smithi151 bash[22565]: audit 2024-09-10T18:33:45.994062+0000 mon.a (mon.0) 621 : audit [DBG] from='client.? 172.21.15.100:0/1347935770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:33:49.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:48 smithi100 bash[18597]: cluster 2024-09-10T18:33:47.279942+0000 mgr.y (mgr.14150) 347 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:49.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:48 smithi100 bash[31607]: cluster 2024-09-10T18:33:47.279942+0000 mgr.y (mgr.14150) 347 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:49.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:48 smithi151 bash[22565]: cluster 2024-09-10T18:33:47.279942+0000 mgr.y (mgr.14150) 347 : cluster [DBG] pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:51.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:50 smithi100 bash[18597]: cluster 2024-09-10T18:33:49.280529+0000 mgr.y (mgr.14150) 348 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:51.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:50 smithi100 bash[31607]: cluster 2024-09-10T18:33:49.280529+0000 mgr.y (mgr.14150) 348 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:51.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:50 smithi151 bash[22565]: cluster 2024-09-10T18:33:49.280529+0000 mgr.y (mgr.14150) 348 : cluster [DBG] pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:51.519 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:33:53.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:52 smithi100 bash[18597]: cluster 2024-09-10T18:33:51.281125+0000 mgr.y (mgr.14150) 349 : cluster [DBG] pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:53.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:52 smithi100 bash[31607]: cluster 2024-09-10T18:33:51.281125+0000 mgr.y (mgr.14150) 349 : cluster [DBG] pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:53.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:52 smithi151 bash[22565]: cluster 2024-09-10T18:33:51.281125+0000 mgr.y (mgr.14150) 349 : cluster [DBG] pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:54.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.868+0000 7fef6fd1c640 1 -- 172.21.15.100:0/582120683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef680fb950 msgr2=0x7fef680fbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:54.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.868+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/582120683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef680fb950 0x7fef680fbdc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fef5c004980 tx=0x7fef5c009da0 comp rx=0 tx=0).stop 2024-09-10T18:33:54.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- 172.21.15.100:0/582120683 shutdown_connections 2024-09-10T18:33:54.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/582120683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef680fc300 0x7fef68108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:54.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/582120683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef680fb950 0x7fef680fbdc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:54.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/582120683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef68101760 0x7fef68101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:54.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- 172.21.15.100:0/582120683 >> 172.21.15.100:0/582120683 conn(0x7fef68072440 msgr2=0x7fef68072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:54.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- 172.21.15.100:0/582120683 shutdown_connections 2024-09-10T18:33:54.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- 172.21.15.100:0/582120683 wait complete. 2024-09-10T18:33:54.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 Processor -- start 2024-09-10T18:33:54.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- start start 2024-09-10T18:33:54.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef680fb950 0x7fef68111c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:54.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef680fc300 0x7fef68112170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:54.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef68101760 0x7fef68065340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:54.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680659e0 con 0x7fef680fc300 2024-09-10T18:33:54.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68065b50 con 0x7fef680fb950 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef68065d10 con 0x7fef68101760 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef68101760 0x7fef68065340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef68101760 0x7fef68065340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:56494/0 (socket says 172.21.15.100:56494) 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 -- 172.21.15.100:0/1982018443 learned_addr learned my addr 172.21.15.100:0/1982018443 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6d2b7640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef680fc300 0x7fef68112170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6dab8640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef680fb950 0x7fef68111c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 -- 172.21.15.100:0/1982018443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef680fb950 msgr2=0x7fef68111c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef680fb950 0x7fef68111c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 -- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef680fc300 msgr2=0x7fef68112170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef680fc300 0x7fef68112170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 -- 172.21.15.100:0/1982018443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c005040 con 0x7fef68101760 2024-09-10T18:33:54.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6dab8640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef680fb950 0x7fef68111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:33:54.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6e2b9640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef68101760 0x7fef68065340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fef6000c980 tx=0x7fef60007db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:33:54.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef5a7fc640 1 -- 172.21.15.100:0/1982018443 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000a850 con 0x7fef68101760 2024-09-10T18:33:54.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef68065ff0 con 0x7fef68101760 2024-09-10T18:33:54.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef68119d70 con 0x7fef68101760 2024-09-10T18:33:54.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef5a7fc640 1 -- 172.21.15.100:0/1982018443 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fef6000ae20 con 0x7fef68101760 2024-09-10T18:33:54.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.872+0000 7fef5a7fc640 1 -- 172.21.15.100:0/1982018443 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60012cb0 con 0x7fef68101760 2024-09-10T18:33:54.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.876+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c005010 con 0x7fef68101760 2024-09-10T18:33:54.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.876+0000 7fef5a7fc640 1 -- 172.21.15.100:0/1982018443 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7fef600115b0 con 0x7fef68101760 2024-09-10T18:33:54.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.876+0000 7fef5a7fc640 1 --2- 172.21.15.100:0/1982018443 >> v2:172.21.15.100:6800/3986579393 conn(0x7fef5406edd0 0x7fef54071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:33:54.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.876+0000 7fef5a7fc640 1 -- 172.21.15.100:0/1982018443 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7fef6008e080 con 0x7fef68101760 2024-09-10T18:33:54.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.876+0000 7fef6dab8640 1 --2- 172.21.15.100:0/1982018443 >> v2:172.21.15.100:6800/3986579393 conn(0x7fef5406edd0 0x7fef54071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:33:54.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.876+0000 7fef6dab8640 1 --2- 172.21.15.100:0/1982018443 >> v2:172.21.15.100:6800/3986579393 conn(0x7fef5406edd0 0x7fef54071280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef64004ab0 tx=0x7fef6400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:33:54.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:54.884+0000 7fef5a7fc640 1 -- 172.21.15.100:0/1982018443 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fef6005b870 con 0x7fef68101760 2024-09-10T18:33:55.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:54 smithi100 bash[18597]: cluster 2024-09-10T18:33:53.281731+0000 mgr.y (mgr.14150) 350 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:55.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:54 smithi100 bash[31607]: cluster 2024-09-10T18:33:53.281731+0000 mgr.y (mgr.14150) 350 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:55.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.060+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef4c005840 con 0x7fef68101760 2024-09-10T18:33:55.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.060+0000 7fef5a7fc640 1 -- 172.21.15.100:0/1982018443 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef6005b210 con 0x7fef68101760 2024-09-10T18:33:55.061 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-09-10T18:33:55.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 >> v2:172.21.15.100:6800/3986579393 conn(0x7fef5406edd0 msgr2=0x7fef54071280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:55.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/1982018443 >> v2:172.21.15.100:6800/3986579393 conn(0x7fef5406edd0 0x7fef54071280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef64004ab0 tx=0x7fef6400b040 comp rx=0 tx=0).stop 2024-09-10T18:33:55.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef68101760 msgr2=0x7fef68065340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:55.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef68101760 0x7fef68065340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fef6000c980 tx=0x7fef60007db0 comp rx=0 tx=0).stop 2024-09-10T18:33:55.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 shutdown_connections 2024-09-10T18:33:55.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/1982018443 >> v2:172.21.15.100:6800/3986579393 conn(0x7fef5406edd0 0x7fef54071280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:55.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef68101760 0x7fef68065340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:55.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef680fc300 0x7fef68112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:55.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 --2- 172.21.15.100:0/1982018443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef680fb950 0x7fef68111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:33:55.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 >> 172.21.15.100:0/1982018443 conn(0x7fef68072440 msgr2=0x7fef680ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:55.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 shutdown_connections 2024-09-10T18:33:55.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:33:55.064+0000 7fef6fd1c640 1 -- 172.21.15.100:0/1982018443 wait complete. 2024-09-10T18:33:55.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:54 smithi151 bash[22565]: cluster 2024-09-10T18:33:53.281731+0000 mgr.y (mgr.14150) 350 : cluster [DBG] pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:55.761 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi100 2024-09-10T18:33:55.761 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi100=foo.a' 2024-09-10T18:33:56.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:55 smithi100 bash[18597]: audit 2024-09-10T18:33:55.062127+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.100:0/1982018443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:33:56.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:55 smithi100 bash[31607]: audit 2024-09-10T18:33:55.062127+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.100:0/1982018443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:33:56.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:55 smithi151 bash[22565]: audit 2024-09-10T18:33:55.062127+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.100:0/1982018443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:33:57.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:56 smithi100 bash[31607]: cluster 2024-09-10T18:33:55.282251+0000 mgr.y (mgr.14150) 351 : cluster [DBG] pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:57.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:56 smithi100 bash[18597]: cluster 2024-09-10T18:33:55.282251+0000 mgr.y (mgr.14150) 351 : cluster [DBG] pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:57.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:56 smithi151 bash[22565]: cluster 2024-09-10T18:33:55.282251+0000 mgr.y (mgr.14150) 351 : cluster [DBG] pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:59.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:33:58 smithi100 bash[18597]: cluster 2024-09-10T18:33:57.282850+0000 mgr.y (mgr.14150) 352 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:59.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:33:58 smithi100 bash[31607]: cluster 2024-09-10T18:33:57.282850+0000 mgr.y (mgr.14150) 352 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:33:59.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:33:58 smithi151 bash[22565]: cluster 2024-09-10T18:33:57.282850+0000 mgr.y (mgr.14150) 352 : cluster [DBG] pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:00.511 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:00.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:00 smithi151 bash[22565]: cluster 2024-09-10T18:33:59.283415+0000 mgr.y (mgr.14150) 353 : cluster [DBG] pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:01.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:00 smithi100 bash[18597]: cluster 2024-09-10T18:33:59.283415+0000 mgr.y (mgr.14150) 353 : cluster [DBG] pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:01.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:00 smithi100 bash[31607]: cluster 2024-09-10T18:33:59.283415+0000 mgr.y (mgr.14150) 353 : cluster [DBG] pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:03.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:02 smithi100 bash[18597]: cluster 2024-09-10T18:34:01.284085+0000 mgr.y (mgr.14150) 354 : cluster [DBG] pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:03.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:02 smithi100 bash[31607]: cluster 2024-09-10T18:34:01.284085+0000 mgr.y (mgr.14150) 354 : cluster [DBG] pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:03.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:02 smithi151 bash[22565]: cluster 2024-09-10T18:34:01.284085+0000 mgr.y (mgr.14150) 354 : cluster [DBG] pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:04.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 -- 172.21.15.151:0/2503973172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480ffcc0 msgr2=0x7fa048100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2503973172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480ffcc0 0x7fa048100130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa040009cd0 tx=0x7fa040004d10 comp rx=0 tx=0).stop 2024-09-10T18:34:04.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 -- 172.21.15.151:0/2503973172 shutdown_connections 2024-09-10T18:34:04.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2503973172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa048100670 0x7fa048106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2503973172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480ffcc0 0x7fa048100130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2503973172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0480fe9f0 0x7fa0480fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 -- 172.21.15.151:0/2503973172 >> 172.21.15.151:0/2503973172 conn(0x7fa0480fa200 msgr2=0x7fa0480fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:04.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 -- 172.21.15.151:0/2503973172 shutdown_connections 2024-09-10T18:34:04.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.982+0000 7fa04fa66640 1 -- 172.21.15.151:0/2503973172 wait complete. 2024-09-10T18:34:04.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 Processor -- start 2024-09-10T18:34:04.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 -- start start 2024-09-10T18:34:04.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480fe9f0 0x7fa04810efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:04.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0480ffcc0 0x7fa04810f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:04.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa048100670 0x7fa048116520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:04.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048116af0 con 0x7fa048100670 2024-09-10T18:34:04.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048116c60 con 0x7fa0480fe9f0 2024-09-10T18:34:04.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa048116e20 con 0x7fa0480ffcc0 2024-09-10T18:34:04.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480fe9f0 0x7fa04810efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480fe9f0 0x7fa04810efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60758/0 (socket says 172.21.15.151:60758) 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 -- 172.21.15.151:0/2426577903 learned_addr learned my addr 172.21.15.151:0/2426577903 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d001640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0480ffcc0 0x7fa04810f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04e003640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa048100670 0x7fa048116520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 -- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0480ffcc0 msgr2=0x7fa04810f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0480ffcc0 0x7fa04810f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 -- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa048100670 msgr2=0x7fa048116520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa048100670 0x7fa048116520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 -- 172.21.15.151:0/2426577903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400098b0 con 0x7fa0480fe9f0 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04e003640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa048100670 0x7fa048116520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04d802640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480fe9f0 0x7fa04810efd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0440028a0 tx=0x7fa044004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:04.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa03a7fc640 1 -- 172.21.15.151:0/2426577903 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044007c30 con 0x7fa0480fe9f0 2024-09-10T18:34:04.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa03a7fc640 1 -- 172.21.15.151:0/2426577903 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fa04400f040 con 0x7fa0480fe9f0 2024-09-10T18:34:04.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa048117100 con 0x7fa0480fe9f0 2024-09-10T18:34:04.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa03a7fc640 1 -- 172.21.15.151:0/2426577903 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044009340 con 0x7fa0480fe9f0 2024-09-10T18:34:04.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.986+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa048117670 con 0x7fa0480fe9f0 2024-09-10T18:34:04.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.990+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04804bf60 con 0x7fa0480fe9f0 2024-09-10T18:34:04.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.990+0000 7fa03a7fc640 1 -- 172.21.15.151:0/2426577903 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7fa0440096f0 con 0x7fa0480fe9f0 2024-09-10T18:34:04.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.990+0000 7fa03a7fc640 1 --2- 172.21.15.151:0/2426577903 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa03406ee00 0x7fa0340712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:04.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.990+0000 7fa03a7fc640 1 -- 172.21.15.151:0/2426577903 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4077+0+0 (secure 0 0 0) 0x7fa04408d9d0 con 0x7fa0480fe9f0 2024-09-10T18:34:04.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.990+0000 7fa04d001640 1 --2- 172.21.15.151:0/2426577903 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa03406ee00 0x7fa0340712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:04.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.994+0000 7fa04d001640 1 --2- 172.21.15.151:0/2426577903 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa03406ee00 0x7fa0340712b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa040004a30 tx=0x7fa040035040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:04.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:03.998+0000 7fa03a7fc640 1 -- 172.21.15.151:0/2426577903 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa04405b0e0 con 0x7fa0480fe9f0 2024-09-10T18:34:04.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.186+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fa04805db50 con 0x7fa03406ee00 2024-09-10T18:34:04.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.198+0000 7fa03a7fc640 1 -- 172.21.15.151:0/2426577903 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fa04805db50 con 0x7fa03406ee00 2024-09-10T18:34:04.217 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled rgw.foo.a update... 2024-09-10T18:34:04.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa03406ee00 msgr2=0x7fa0340712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2426577903 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa03406ee00 0x7fa0340712b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa040004a30 tx=0x7fa040035040 comp rx=0 tx=0).stop 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480fe9f0 msgr2=0x7fa04810efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480fe9f0 0x7fa04810efd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa0440028a0 tx=0x7fa044004570 comp rx=0 tx=0).stop 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 shutdown_connections 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2426577903 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa03406ee00 0x7fa0340712b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa048100670 0x7fa048116520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0480ffcc0 0x7fa04810f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 --2- 172.21.15.151:0/2426577903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0480fe9f0 0x7fa04810efd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:04.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 >> 172.21.15.151:0/2426577903 conn(0x7fa0480fa200 msgr2=0x7fa0480fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:04.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 shutdown_connections 2024-09-10T18:34:04.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:04.202+0000 7fa04fa66640 1 -- 172.21.15.151:0/2426577903 wait complete. 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: cluster 2024-09-10T18:34:03.284721+0000 mgr.y (mgr.14150) 355 : cluster [DBG] pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.216876+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.218523+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.221729+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.223383+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.232284+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.236228+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.242394+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:34:05.040 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.250842+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.041 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:04 smithi151 bash[22565]: audit 2024-09-10T18:34:04.252256+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:05.041 DEBUG:teuthology.orchestra.run.smithi100:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: cluster 2024-09-10T18:34:03.284721+0000 mgr.y (mgr.14150) 355 : cluster [DBG] pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.216876+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.218523+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.221729+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.223383+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.232284+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.236228+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.242394+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.250842+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[31607]: audit 2024-09-10T18:34:04.252256+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: cluster 2024-09-10T18:34:03.284721+0000 mgr.y (mgr.14150) 355 : cluster [DBG] pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.216876+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.218523+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.221729+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.223383+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.232284+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.236228+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.242394+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.250842+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:05.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:04 smithi100 bash[18597]: audit 2024-09-10T18:34:04.252256+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:05.045 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi151 2024-09-10T18:34:05.045 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T18:34:06.030 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:34:06.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:05 smithi100 bash[18597]: audit 2024-09-10T18:34:04.205926+0000 mgr.y (mgr.14150) 356 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:06.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:05 smithi100 bash[18597]: cephadm 2024-09-10T18:34:04.209408+0000 mgr.y (mgr.14150) 357 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-09-10T18:34:06.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:05 smithi100 bash[18597]: cephadm 2024-09-10T18:34:04.253903+0000 mgr.y (mgr.14150) 358 : cephadm [INF] Deploying daemon rgw.foo.a on smithi100 2024-09-10T18:34:06.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:05 smithi100 bash[31607]: audit 2024-09-10T18:34:04.205926+0000 mgr.y (mgr.14150) 356 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:06.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:05 smithi100 bash[31607]: cephadm 2024-09-10T18:34:04.209408+0000 mgr.y (mgr.14150) 357 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-09-10T18:34:06.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:05 smithi100 bash[31607]: cephadm 2024-09-10T18:34:04.253903+0000 mgr.y (mgr.14150) 358 : cephadm [INF] Deploying daemon rgw.foo.a on smithi100 2024-09-10T18:34:06.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:05 smithi151 bash[22565]: audit 2024-09-10T18:34:04.205926+0000 mgr.y (mgr.14150) 356 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:06.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:05 smithi151 bash[22565]: cephadm 2024-09-10T18:34:04.209408+0000 mgr.y (mgr.14150) 357 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-09-10T18:34:06.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:05 smithi151 bash[22565]: cephadm 2024-09-10T18:34:04.253903+0000 mgr.y (mgr.14150) 358 : cephadm [INF] Deploying daemon rgw.foo.a on smithi100 2024-09-10T18:34:07.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:06 smithi100 bash[18597]: cluster 2024-09-10T18:34:05.285272+0000 mgr.y (mgr.14150) 359 : cluster [DBG] pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:07.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:06 smithi100 bash[31607]: cluster 2024-09-10T18:34:05.285272+0000 mgr.y (mgr.14150) 359 : cluster [DBG] pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:07.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:06 smithi151 bash[22565]: cluster 2024-09-10T18:34:05.285272+0000 mgr.y (mgr.14150) 359 : cluster [DBG] pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:07.461 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Sep 10 18:34:07 smithi100 systemd[1]: Started Ceph rgw.foo.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: cluster 2024-09-10T18:34:07.285853+0000 mgr.y (mgr.14150) 360 : cluster [DBG] pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.359357+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.367205+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.373608+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: cephadm 2024-09-10T18:34:07.374509+0000 mgr.y (mgr.14150) 361 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.380824+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.407185+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.449793+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:08.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.733571+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[18597]: audit 2024-09-10T18:34:07.740433+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: cluster 2024-09-10T18:34:07.285853+0000 mgr.y (mgr.14150) 360 : cluster [DBG] pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.359357+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.367205+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.373608+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: cephadm 2024-09-10T18:34:07.374509+0000 mgr.y (mgr.14150) 361 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.380824+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.407185+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.449793+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.733571+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.631 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:08 smithi100 bash[31607]: audit 2024-09-10T18:34:07.740433+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: cluster 2024-09-10T18:34:07.285853+0000 mgr.y (mgr.14150) 360 : cluster [DBG] pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:08.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.359357+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.367205+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.373608+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: cephadm 2024-09-10T18:34:07.374509+0000 mgr.y (mgr.14150) 361 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-09-10T18:34:08.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.380824+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.407185+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.449793+0000 mon.a (mon.0) 636 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:08.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.733571+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:08.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:08 smithi151 bash[22565]: audit 2024-09-10T18:34:07.740433+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:09.796 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:10.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:10 smithi151 bash[22565]: cluster 2024-09-10T18:34:09.286464+0000 mgr.y (mgr.14150) 362 : cluster [DBG] pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:10.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:10 smithi151 bash[22565]: cluster 2024-09-10T18:34:09.372128+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:34:10.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:10 smithi151 bash[22565]: audit 2024-09-10T18:34:09.373464+0000 mon.a (mon.0) 640 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:34:10.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:10 smithi100 bash[31607]: cluster 2024-09-10T18:34:09.286464+0000 mgr.y (mgr.14150) 362 : cluster [DBG] pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:10.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:10 smithi100 bash[31607]: cluster 2024-09-10T18:34:09.372128+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:34:10.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:10 smithi100 bash[31607]: audit 2024-09-10T18:34:09.373464+0000 mon.a (mon.0) 640 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:34:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:10 smithi100 bash[18597]: cluster 2024-09-10T18:34:09.286464+0000 mgr.y (mgr.14150) 362 : cluster [DBG] pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:10 smithi100 bash[18597]: cluster 2024-09-10T18:34:09.372128+0000 mon.a (mon.0) 639 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:34:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:10 smithi100 bash[18597]: audit 2024-09-10T18:34:09.373464+0000 mon.a (mon.0) 640 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:34:11.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:11 smithi151 bash[22565]: audit 2024-09-10T18:34:10.378222+0000 mon.a (mon.0) 641 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:34:11.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:11 smithi151 bash[22565]: cluster 2024-09-10T18:34:10.378771+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:34:11.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:11 smithi151 bash[22565]: audit 2024-09-10T18:34:10.912421+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:11 smithi100 bash[18597]: audit 2024-09-10T18:34:10.378222+0000 mon.a (mon.0) 641 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:34:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:11 smithi100 bash[18597]: cluster 2024-09-10T18:34:10.378771+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:34:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:11 smithi100 bash[18597]: audit 2024-09-10T18:34:10.912421+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:11.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:11 smithi100 bash[31607]: audit 2024-09-10T18:34:10.378222+0000 mon.a (mon.0) 641 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:34:11.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:11 smithi100 bash[31607]: cluster 2024-09-10T18:34:10.378771+0000 mon.a (mon.0) 642 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:34:11.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:11 smithi100 bash[31607]: audit 2024-09-10T18:34:10.912421+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:12.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:12 smithi151 bash[22565]: cluster 2024-09-10T18:34:11.287135+0000 mgr.y (mgr.14150) 363 : cluster [DBG] pgmap v339: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:12.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:12 smithi151 bash[22565]: cluster 2024-09-10T18:34:11.408935+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:34:12.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:12 smithi151 bash[22565]: audit 2024-09-10T18:34:11.412195+0000 mon.a (mon.0) 645 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:34:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:12 smithi100 bash[18597]: cluster 2024-09-10T18:34:11.287135+0000 mgr.y (mgr.14150) 363 : cluster [DBG] pgmap v339: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:12 smithi100 bash[18597]: cluster 2024-09-10T18:34:11.408935+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:34:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:12 smithi100 bash[18597]: audit 2024-09-10T18:34:11.412195+0000 mon.a (mon.0) 645 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:34:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:12 smithi100 bash[31607]: cluster 2024-09-10T18:34:11.287135+0000 mgr.y (mgr.14150) 363 : cluster [DBG] pgmap v339: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:12 smithi100 bash[31607]: cluster 2024-09-10T18:34:11.408935+0000 mon.a (mon.0) 644 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:34:12.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:12 smithi100 bash[31607]: audit 2024-09-10T18:34:11.412195+0000 mon.a (mon.0) 645 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:34:13.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- 172.21.15.151:0/3581305167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 msgr2=0x7fa2b00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:13.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/3581305167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b00fce40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4004950 tx=0x7fa2a4009da0 comp rx=0 tx=0).stop 2024-09-10T18:34:13.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- 172.21.15.151:0/3581305167 shutdown_connections 2024-09-10T18:34:13.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/3581305167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b00fd380 0x7fa2b0103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:13.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/3581305167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b00fce40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:13.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/3581305167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b00fb700 0x7fa2b00fbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:13.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- 172.21.15.151:0/3581305167 >> 172.21.15.151:0/3581305167 conn(0x7fa2b00f6ea0 msgr2=0x7fa2b00f92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:13.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- 172.21.15.151:0/3581305167 shutdown_connections 2024-09-10T18:34:13.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- 172.21.15.151:0/3581305167 wait complete. 2024-09-10T18:34:13.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 Processor -- start 2024-09-10T18:34:13.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- start start 2024-09-10T18:34:13.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b00fb700 0x7fa2b010ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:13.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b010f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:13.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b00fd380 0x7fa2b01164d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:13.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0116aa0 con 0x7fa2b00fb700 2024-09-10T18:34:13.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0116c10 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0116dd0 con 0x7fa2b00fd380 2024-09-10T18:34:13.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b010f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:13.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b010f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34786/0 (socket says 172.21.15.151:34786) 2024-09-10T18:34:13.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 -- 172.21.15.151:0/1170227636 learned_addr learned my addr 172.21.15.151:0/1170227636 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:34:13.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2af7fe640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b00fd380 0x7fa2b01164d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:13.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2aeffd640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b00fb700 0x7fa2b010ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:13.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 -- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b00fd380 msgr2=0x7fa2b01164d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:13.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b00fd380 0x7fa2b01164d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:13.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 -- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b00fb700 msgr2=0x7fa2b010ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:13.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b00fb700 0x7fa2b010ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:13.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 -- 172.21.15.151:0/1170227636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a4005040 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2aeffd640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b00fb700 0x7fa2b010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:34:13.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2ae7fc640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b010f470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2a402d040 tx=0x7fa2a4035040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:13.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2977fe640 1 -- 172.21.15.151:0/1170227636 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a402ab70 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2977fe640 1 -- 172.21.15.151:0/1170227636 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fa2a402acd0 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b0117050 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2977fe640 1 -- 172.21.15.151:0/1170227636 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4033670 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.338+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b01175f0 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.342+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c005010 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.342+0000 7fa2977fe640 1 -- 172.21.15.151:0/1170227636 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7fa2a4030020 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.342+0000 7fa2977fe640 1 --2- 172.21.15.151:0/1170227636 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa29806ee00 0x7fa2980712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:13.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.342+0000 7fa2977fe640 1 -- 172.21.15.151:0/1170227636 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4801+0+0 (secure 0 0 0) 0x7fa2a40ae090 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.342+0000 7fa2aeffd640 1 --2- 172.21.15.151:0/1170227636 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa29806ee00 0x7fa2980712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:13.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.346+0000 7fa2aeffd640 1 --2- 172.21.15.151:0/1170227636 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa29806ee00 0x7fa2980712b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00041d0 tx=0x7fa2a0009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:13.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.350+0000 7fa2977fe640 1 -- 172.21.15.151:0/1170227636 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa2a407b5a0 con 0x7fa2b00fc9d0 2024-09-10T18:34:13.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:13.526+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fa29c005590 con 0x7fa2b00fc9d0 2024-09-10T18:34:14.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[31607]: audit 2024-09-10T18:34:12.409523+0000 mon.a (mon.0) 646 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:34:14.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[31607]: cluster 2024-09-10T18:34:12.409654+0000 mon.a (mon.0) 647 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:34:14.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[31607]: audit 2024-09-10T18:34:13.013578+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[31607]: audit 2024-09-10T18:34:13.021440+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[31607]: audit 2024-09-10T18:34:13.022713+0000 mon.a (mon.0) 650 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[31607]: audit 2024-09-10T18:34:13.023608+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[18597]: audit 2024-09-10T18:34:12.409523+0000 mon.a (mon.0) 646 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[18597]: cluster 2024-09-10T18:34:12.409654+0000 mon.a (mon.0) 647 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[18597]: audit 2024-09-10T18:34:13.013578+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[18597]: audit 2024-09-10T18:34:13.021440+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[18597]: audit 2024-09-10T18:34:13.022713+0000 mon.a (mon.0) 650 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:13 smithi100 bash[18597]: audit 2024-09-10T18:34:13.023608+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:14.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:13 smithi151 bash[22565]: audit 2024-09-10T18:34:12.409523+0000 mon.a (mon.0) 646 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:34:14.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:13 smithi151 bash[22565]: cluster 2024-09-10T18:34:12.409654+0000 mon.a (mon.0) 647 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:34:14.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:13 smithi151 bash[22565]: audit 2024-09-10T18:34:13.013578+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:14.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:13 smithi151 bash[22565]: audit 2024-09-10T18:34:13.021440+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:14.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:13 smithi151 bash[22565]: audit 2024-09-10T18:34:13.022713+0000 mon.a (mon.0) 650 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:14.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:13 smithi151 bash[22565]: audit 2024-09-10T18:34:13.023608+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:14.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.738+0000 7fa2977fe640 1 -- 172.21.15.151:0/1170227636 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v57) v1 ==== 160+0+0 (secure 0 0 0) 0x7fa2a407af40 con 0x7fa2b00fc9d0 2024-09-10T18:34:14.758 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'datapool' created 2024-09-10T18:34:14.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa29806ee00 msgr2=0x7fa2980712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:14.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/1170227636 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa29806ee00 0x7fa2980712b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00041d0 tx=0x7fa2a0009040 comp rx=0 tx=0).stop 2024-09-10T18:34:14.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 msgr2=0x7fa2b010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b010f470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2a402d040 tx=0x7fa2a4035040 comp rx=0 tx=0).stop 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 shutdown_connections 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/1170227636 >> v2:172.21.15.100:6800/3986579393 conn(0x7fa29806ee00 0x7fa2980712b0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00041d0 tx=0x7fa2a0009040 comp rx=0 tx=0).stop 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b00fd380 0x7fa2b01164d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b00fc9d0 0x7fa2b010f470 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2a402d040 tx=0x7fa2a4035040 comp rx=0 tx=0).stop 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 --2- 172.21.15.151:0/1170227636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b00fb700 0x7fa2b010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 >> 172.21.15.151:0/1170227636 conn(0x7fa2b00f6ea0 msgr2=0x7fa2b0065fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 shutdown_connections 2024-09-10T18:34:14.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:14.746+0000 7fa2b56a1640 1 -- 172.21.15.151:0/1170227636 wait complete. 2024-09-10T18:34:15.033 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: cephadm 2024-09-10T18:34:13.029311+0000 mgr.y (mgr.14150) 364 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-09-10T18:34:15.033 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: cluster 2024-09-10T18:34:13.288425+0000 mgr.y (mgr.14150) 365 : cluster [DBG] pgmap v342: 65 pgs: 25 active+clean, 16 creating+peering, 24 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:34:15.034 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: audit 2024-09-10T18:34:13.531222+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.151:0/1170227636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:34:15.034 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: audit 2024-09-10T18:34:13.740050+0000 mon.a (mon.0) 652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:34:15.034 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: audit 2024-09-10T18:34:13.752960+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.100:0/1699018528' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.034 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: cluster 2024-09-10T18:34:14.149089+0000 mon.a (mon.0) 653 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:34:15.034 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: audit 2024-09-10T18:34:14.150822+0000 mon.a (mon.0) 654 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.034 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:14 smithi151 bash[22565]: audit 2024-09-10T18:34:14.155390+0000 mon.a (mon.0) 655 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: cephadm 2024-09-10T18:34:13.029311+0000 mgr.y (mgr.14150) 364 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: cluster 2024-09-10T18:34:13.288425+0000 mgr.y (mgr.14150) 365 : cluster [DBG] pgmap v342: 65 pgs: 25 active+clean, 16 creating+peering, 24 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: audit 2024-09-10T18:34:13.531222+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.151:0/1170227636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: audit 2024-09-10T18:34:13.740050+0000 mon.a (mon.0) 652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: audit 2024-09-10T18:34:13.752960+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.100:0/1699018528' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: cluster 2024-09-10T18:34:14.149089+0000 mon.a (mon.0) 653 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: audit 2024-09-10T18:34:14.150822+0000 mon.a (mon.0) 654 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[31607]: audit 2024-09-10T18:34:14.155390+0000 mon.a (mon.0) 655 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: cephadm 2024-09-10T18:34:13.029311+0000 mgr.y (mgr.14150) 364 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: cluster 2024-09-10T18:34:13.288425+0000 mgr.y (mgr.14150) 365 : cluster [DBG] pgmap v342: 65 pgs: 25 active+clean, 16 creating+peering, 24 unknown; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: audit 2024-09-10T18:34:13.531222+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.151:0/1170227636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: audit 2024-09-10T18:34:13.740050+0000 mon.a (mon.0) 652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: audit 2024-09-10T18:34:13.752960+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.100:0/1699018528' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: cluster 2024-09-10T18:34:14.149089+0000 mon.a (mon.0) 653 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: audit 2024-09-10T18:34:14.150822+0000 mon.a (mon.0) 654 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:14 smithi100 bash[18597]: audit 2024-09-10T18:34:14.155390+0000 mon.a (mon.0) 655 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:34:15.608 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T18:34:15.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:15 smithi151 bash[22565]: audit 2024-09-10T18:34:14.748901+0000 mon.a (mon.0) 656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:34:15.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:15 smithi151 bash[22565]: audit 2024-09-10T18:34:14.749165+0000 mon.a (mon.0) 657 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:34:15.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:15 smithi151 bash[22565]: audit 2024-09-10T18:34:14.749351+0000 mon.a (mon.0) 658 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:34:15.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:15 smithi151 bash[22565]: cluster 2024-09-10T18:34:14.749501+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:34:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[31607]: audit 2024-09-10T18:34:14.748901+0000 mon.a (mon.0) 656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:34:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[31607]: audit 2024-09-10T18:34:14.749165+0000 mon.a (mon.0) 657 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:34:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[31607]: audit 2024-09-10T18:34:14.749351+0000 mon.a (mon.0) 658 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:34:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[31607]: cluster 2024-09-10T18:34:14.749501+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:34:16.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[18597]: audit 2024-09-10T18:34:14.748901+0000 mon.a (mon.0) 656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:34:16.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[18597]: audit 2024-09-10T18:34:14.749165+0000 mon.a (mon.0) 657 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:34:16.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[18597]: audit 2024-09-10T18:34:14.749351+0000 mon.a (mon.0) 658 : audit [INF] from='client.? 172.21.15.100:0/747850926' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:34:16.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:15 smithi100 bash[18597]: cluster 2024-09-10T18:34:14.749501+0000 mon.a (mon.0) 659 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:34:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[18597]: cluster 2024-09-10T18:34:15.289394+0000 mgr.y (mgr.14150) 366 : cluster [DBG] pgmap v345: 100 pgs: 74 active+clean, 10 creating+peering, 16 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:34:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[18597]: cluster 2024-09-10T18:34:15.744501+0000 mon.a (mon.0) 660 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:34:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[18597]: audit 2024-09-10T18:34:15.752467+0000 mon.b (mon.1) 27 : audit [INF] from='client.? 172.21.15.100:0/3844752144' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[18597]: cluster 2024-09-10T18:34:15.764108+0000 mon.a (mon.0) 661 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:34:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[18597]: audit 2024-09-10T18:34:15.768084+0000 mon.a (mon.0) 662 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[18597]: audit 2024-09-10T18:34:15.768572+0000 mon.a (mon.0) 663 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[31607]: cluster 2024-09-10T18:34:15.289394+0000 mgr.y (mgr.14150) 366 : cluster [DBG] pgmap v345: 100 pgs: 74 active+clean, 10 creating+peering, 16 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:34:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[31607]: cluster 2024-09-10T18:34:15.744501+0000 mon.a (mon.0) 660 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:34:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[31607]: audit 2024-09-10T18:34:15.752467+0000 mon.b (mon.1) 27 : audit [INF] from='client.? 172.21.15.100:0/3844752144' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[31607]: cluster 2024-09-10T18:34:15.764108+0000 mon.a (mon.0) 661 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:34:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[31607]: audit 2024-09-10T18:34:15.768084+0000 mon.a (mon.0) 662 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:16 smithi100 bash[31607]: audit 2024-09-10T18:34:15.768572+0000 mon.a (mon.0) 663 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:16 smithi151 bash[22565]: cluster 2024-09-10T18:34:15.289394+0000 mgr.y (mgr.14150) 366 : cluster [DBG] pgmap v345: 100 pgs: 74 active+clean, 10 creating+peering, 16 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:34:17.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:16 smithi151 bash[22565]: cluster 2024-09-10T18:34:15.744501+0000 mon.a (mon.0) 660 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:34:17.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:16 smithi151 bash[22565]: audit 2024-09-10T18:34:15.752467+0000 mon.b (mon.1) 27 : audit [INF] from='client.? 172.21.15.100:0/3844752144' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:16 smithi151 bash[22565]: cluster 2024-09-10T18:34:15.764108+0000 mon.a (mon.0) 661 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:34:17.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:16 smithi151 bash[22565]: audit 2024-09-10T18:34:15.768084+0000 mon.a (mon.0) 662 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:17.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:16 smithi151 bash[22565]: audit 2024-09-10T18:34:15.768572+0000 mon.a (mon.0) 663 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[18597]: audit 2024-09-10T18:34:16.768173+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.100:0/3844752144' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[18597]: audit 2024-09-10T18:34:16.778246+0000 mon.a (mon.0) 664 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[18597]: audit 2024-09-10T18:34:16.778387+0000 mon.a (mon.0) 665 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[18597]: cluster 2024-09-10T18:34:16.778541+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[18597]: audit 2024-09-10T18:34:16.782009+0000 mon.a (mon.0) 667 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[18597]: audit 2024-09-10T18:34:16.783625+0000 mon.a (mon.0) 668 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[31607]: audit 2024-09-10T18:34:16.768173+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.100:0/3844752144' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[31607]: audit 2024-09-10T18:34:16.778246+0000 mon.a (mon.0) 664 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:34:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[31607]: audit 2024-09-10T18:34:16.778387+0000 mon.a (mon.0) 665 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:34:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[31607]: cluster 2024-09-10T18:34:16.778541+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:34:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[31607]: audit 2024-09-10T18:34:16.782009+0000 mon.a (mon.0) 667 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:17 smithi100 bash[31607]: audit 2024-09-10T18:34:16.783625+0000 mon.a (mon.0) 668 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:17 smithi151 bash[22565]: audit 2024-09-10T18:34:16.768173+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.100:0/3844752144' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:17 smithi151 bash[22565]: audit 2024-09-10T18:34:16.778246+0000 mon.a (mon.0) 664 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:34:18.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:17 smithi151 bash[22565]: audit 2024-09-10T18:34:16.778387+0000 mon.a (mon.0) 665 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:34:18.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:17 smithi151 bash[22565]: cluster 2024-09-10T18:34:16.778541+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:34:18.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:17 smithi151 bash[22565]: audit 2024-09-10T18:34:16.782009+0000 mon.a (mon.0) 667 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:18.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:17 smithi151 bash[22565]: audit 2024-09-10T18:34:16.783625+0000 mon.a (mon.0) 668 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:34:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: cluster 2024-09-10T18:34:17.290128+0000 mgr.y (mgr.14150) 367 : cluster [DBG] pgmap v348: 132 pgs: 80 active+clean, 10 creating+peering, 42 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:34:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:17.785087+0000 mon.a (mon.0) 669 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:34:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:17.785284+0000 mon.a (mon.0) 670 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:34:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: cluster 2024-09-10T18:34:17.785476+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:34:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:18.013464+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:18.028314+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:18.036443+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:18.096060+0000 mon.a (mon.0) 675 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:18.505714+0000 mon.a (mon.0) 676 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[18597]: audit 2024-09-10T18:34:18.507182+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: cluster 2024-09-10T18:34:17.290128+0000 mgr.y (mgr.14150) 367 : cluster [DBG] pgmap v348: 132 pgs: 80 active+clean, 10 creating+peering, 42 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:34:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:17.785087+0000 mon.a (mon.0) 669 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:34:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:17.785284+0000 mon.a (mon.0) 670 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:34:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: cluster 2024-09-10T18:34:17.785476+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:34:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:18.013464+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:18.028314+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:18.036443+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:18.096060+0000 mon.a (mon.0) 675 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:18.505714+0000 mon.a (mon.0) 676 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:18 smithi100 bash[31607]: audit 2024-09-10T18:34:18.507182+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: cluster 2024-09-10T18:34:17.290128+0000 mgr.y (mgr.14150) 367 : cluster [DBG] pgmap v348: 132 pgs: 80 active+clean, 10 creating+peering, 42 unknown; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:34:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:17.785087+0000 mon.a (mon.0) 669 : audit [INF] from='client.? 172.21.15.100:0/2484365799' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:34:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:17.785284+0000 mon.a (mon.0) 670 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:34:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: cluster 2024-09-10T18:34:17.785476+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:34:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:18.013464+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:18.028314+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:18.036443+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:19.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:18.096060+0000 mon.a (mon.0) 675 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:19.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:18.505714+0000 mon.a (mon.0) 676 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:19.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:18 smithi151 bash[22565]: audit 2024-09-10T18:34:18.507182+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:20.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:19 smithi151 bash[22565]: cephadm 2024-09-10T18:34:18.514668+0000 mgr.y (mgr.14150) 368 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-09-10T18:34:20.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:19 smithi151 bash[22565]: audit 2024-09-10T18:34:18.863969+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:20.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:19 smithi100 bash[18597]: cephadm 2024-09-10T18:34:18.514668+0000 mgr.y (mgr.14150) 368 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-09-10T18:34:20.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:19 smithi100 bash[18597]: audit 2024-09-10T18:34:18.863969+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:20.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:19 smithi100 bash[31607]: cephadm 2024-09-10T18:34:18.514668+0000 mgr.y (mgr.14150) 368 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-09-10T18:34:20.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:19 smithi100 bash[31607]: audit 2024-09-10T18:34:18.863969+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:20.355 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:21.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:20 smithi151 bash[22565]: cluster 2024-09-10T18:34:19.291515+0000 mgr.y (mgr.14150) 369 : cluster [DBG] pgmap v350: 132 pgs: 112 active+clean, 8 creating+peering, 12 unknown; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 40 KiB/s rd, 4.2 KiB/s wr, 90 op/s 2024-09-10T18:34:21.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:20 smithi100 bash[18597]: cluster 2024-09-10T18:34:19.291515+0000 mgr.y (mgr.14150) 369 : cluster [DBG] pgmap v350: 132 pgs: 112 active+clean, 8 creating+peering, 12 unknown; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 40 KiB/s rd, 4.2 KiB/s wr, 90 op/s 2024-09-10T18:34:21.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:20 smithi100 bash[31607]: cluster 2024-09-10T18:34:19.291515+0000 mgr.y (mgr.14150) 369 : cluster [DBG] pgmap v350: 132 pgs: 112 active+clean, 8 creating+peering, 12 unknown; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 40 KiB/s rd, 4.2 KiB/s wr, 90 op/s 2024-09-10T18:34:23.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:22 smithi151 bash[22565]: cluster 2024-09-10T18:34:21.292616+0000 mgr.y (mgr.14150) 370 : cluster [DBG] pgmap v351: 132 pgs: 126 active+clean, 6 creating+peering; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.8 KiB/s wr, 137 op/s 2024-09-10T18:34:23.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:22 smithi100 bash[18597]: cluster 2024-09-10T18:34:21.292616+0000 mgr.y (mgr.14150) 370 : cluster [DBG] pgmap v351: 132 pgs: 126 active+clean, 6 creating+peering; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.8 KiB/s wr, 137 op/s 2024-09-10T18:34:23.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:22 smithi100 bash[31607]: cluster 2024-09-10T18:34:21.292616+0000 mgr.y (mgr.14150) 370 : cluster [DBG] pgmap v351: 132 pgs: 126 active+clean, 6 creating+peering; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.8 KiB/s wr, 137 op/s 2024-09-10T18:34:25.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:24 smithi151 bash[22565]: cluster 2024-09-10T18:34:23.293825+0000 mgr.y (mgr.14150) 371 : cluster [DBG] pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 4.8 KiB/s wr, 153 op/s 2024-09-10T18:34:25.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:24 smithi151 bash[22565]: audit 2024-09-10T18:34:23.940424+0000 mon.a (mon.0) 679 : audit [INF] from='client.? 172.21.15.151:0/3904759560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:34:25.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:24 smithi100 bash[31607]: cluster 2024-09-10T18:34:23.293825+0000 mgr.y (mgr.14150) 371 : cluster [DBG] pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 4.8 KiB/s wr, 153 op/s 2024-09-10T18:34:25.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:24 smithi100 bash[31607]: audit 2024-09-10T18:34:23.940424+0000 mon.a (mon.0) 679 : audit [INF] from='client.? 172.21.15.151:0/3904759560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:34:25.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:24 smithi100 bash[18597]: cluster 2024-09-10T18:34:23.293825+0000 mgr.y (mgr.14150) 371 : cluster [DBG] pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 4.8 KiB/s wr, 153 op/s 2024-09-10T18:34:25.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:24 smithi100 bash[18597]: audit 2024-09-10T18:34:23.940424+0000 mon.a (mon.0) 679 : audit [INF] from='client.? 172.21.15.151:0/3904759560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:34:26.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:25 smithi151 bash[22565]: audit 2024-09-10T18:34:24.897951+0000 mon.a (mon.0) 680 : audit [INF] from='client.? 172.21.15.151:0/3904759560' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:34:26.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:25 smithi151 bash[22565]: cluster 2024-09-10T18:34:24.898104+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:34:26.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:25 smithi100 bash[31607]: audit 2024-09-10T18:34:24.897951+0000 mon.a (mon.0) 680 : audit [INF] from='client.? 172.21.15.151:0/3904759560' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:34:26.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:25 smithi100 bash[31607]: cluster 2024-09-10T18:34:24.898104+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:34:26.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:25 smithi100 bash[18597]: audit 2024-09-10T18:34:24.897951+0000 mon.a (mon.0) 680 : audit [INF] from='client.? 172.21.15.151:0/3904759560' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:34:26.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:25 smithi100 bash[18597]: cluster 2024-09-10T18:34:24.898104+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:34:27.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:26 smithi151 bash[22565]: cluster 2024-09-10T18:34:25.294888+0000 mgr.y (mgr.14150) 372 : cluster [DBG] pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 4.5 KiB/s wr, 144 op/s 2024-09-10T18:34:27.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:26 smithi151 bash[22565]: cluster 2024-09-10T18:34:25.890600+0000 mon.a (mon.0) 682 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:34:27.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:26 smithi151 bash[22565]: cluster 2024-09-10T18:34:25.890665+0000 mon.a (mon.0) 683 : cluster [INF] Cluster is now healthy 2024-09-10T18:34:27.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:26 smithi151 bash[22565]: cluster 2024-09-10T18:34:25.911133+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:34:27.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[18597]: cluster 2024-09-10T18:34:25.294888+0000 mgr.y (mgr.14150) 372 : cluster [DBG] pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 4.5 KiB/s wr, 144 op/s 2024-09-10T18:34:27.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[18597]: cluster 2024-09-10T18:34:25.890600+0000 mon.a (mon.0) 682 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:34:27.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[18597]: cluster 2024-09-10T18:34:25.890665+0000 mon.a (mon.0) 683 : cluster [INF] Cluster is now healthy 2024-09-10T18:34:27.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[18597]: cluster 2024-09-10T18:34:25.911133+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:34:27.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[31607]: cluster 2024-09-10T18:34:25.294888+0000 mgr.y (mgr.14150) 372 : cluster [DBG] pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 4.5 KiB/s wr, 144 op/s 2024-09-10T18:34:27.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[31607]: cluster 2024-09-10T18:34:25.890600+0000 mon.a (mon.0) 682 : cluster [INF] Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:34:27.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[31607]: cluster 2024-09-10T18:34:25.890665+0000 mon.a (mon.0) 683 : cluster [INF] Cluster is now healthy 2024-09-10T18:34:27.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:26 smithi100 bash[31607]: cluster 2024-09-10T18:34:25.911133+0000 mon.a (mon.0) 684 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:34:27.709 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.151 --placement '1;smithi151=iscsi.a' 2024-09-10T18:34:28.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:27 smithi151 bash[22565]: cluster 2024-09-10T18:34:26.915023+0000 mon.a (mon.0) 685 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:34:28.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:27 smithi100 bash[18597]: cluster 2024-09-10T18:34:26.915023+0000 mon.a (mon.0) 685 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:34:28.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:27 smithi100 bash[31607]: cluster 2024-09-10T18:34:26.915023+0000 mon.a (mon.0) 685 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:34:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:28 smithi100 bash[18597]: cluster 2024-09-10T18:34:27.295760+0000 mgr.y (mgr.14150) 373 : cluster [DBG] pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 55 op/s 2024-09-10T18:34:29.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:28 smithi100 bash[31607]: cluster 2024-09-10T18:34:27.295760+0000 mgr.y (mgr.14150) 373 : cluster [DBG] pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 55 op/s 2024-09-10T18:34:29.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:28 smithi151 bash[22565]: cluster 2024-09-10T18:34:27.295760+0000 mgr.y (mgr.14150) 373 : cluster [DBG] pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 55 op/s 2024-09-10T18:34:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:30 smithi100 bash[18597]: cluster 2024-09-10T18:34:29.296890+0000 mgr.y (mgr.14150) 374 : cluster [DBG] pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T18:34:31.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:30 smithi100 bash[31607]: cluster 2024-09-10T18:34:29.296890+0000 mgr.y (mgr.14150) 374 : cluster [DBG] pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T18:34:31.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:30 smithi151 bash[22565]: cluster 2024-09-10T18:34:29.296890+0000 mgr.y (mgr.14150) 374 : cluster [DBG] pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T18:34:32.456 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:33.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:32 smithi100 bash[18597]: cluster 2024-09-10T18:34:31.297993+0000 mgr.y (mgr.14150) 375 : cluster [DBG] pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 319 B/s wr, 0 op/s 2024-09-10T18:34:33.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:32 smithi100 bash[31607]: cluster 2024-09-10T18:34:31.297993+0000 mgr.y (mgr.14150) 375 : cluster [DBG] pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 319 B/s wr, 0 op/s 2024-09-10T18:34:33.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:32 smithi151 bash[22565]: cluster 2024-09-10T18:34:31.297993+0000 mgr.y (mgr.14150) 375 : cluster [DBG] pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 319 B/s wr, 0 op/s 2024-09-10T18:34:35.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:34 smithi151 bash[22565]: cluster 2024-09-10T18:34:33.299070+0000 mgr.y (mgr.14150) 376 : cluster [DBG] pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:34:35.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:34 smithi151 bash[22565]: cluster 2024-09-10T18:34:34.201578+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:34:35.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:34 smithi100 bash[18597]: cluster 2024-09-10T18:34:33.299070+0000 mgr.y (mgr.14150) 376 : cluster [DBG] pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:34:35.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:34 smithi100 bash[18597]: cluster 2024-09-10T18:34:34.201578+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:34:35.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:34 smithi100 bash[31607]: cluster 2024-09-10T18:34:33.299070+0000 mgr.y (mgr.14150) 376 : cluster [DBG] pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:34:35.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:34 smithi100 bash[31607]: cluster 2024-09-10T18:34:34.201578+0000 mon.a (mon.0) 686 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:34:36.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- 172.21.15.151:0/3141505043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fca10 msgr2=0x7f2e0c0fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:36.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 --2- 172.21.15.151:0/3141505043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fca10 0x7f2e0c0fce80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009cd0 tx=0x7f2df4004d10 comp rx=0 tx=0).stop 2024-09-10T18:34:36.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- 172.21.15.151:0/3141505043 shutdown_connections 2024-09-10T18:34:36.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 --2- 172.21.15.151:0/3141505043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e0c0fd3c0 0x7f2e0c103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 --2- 172.21.15.151:0/3141505043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fca10 0x7f2e0c0fce80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 --2- 172.21.15.151:0/3141505043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0fb740 0x7f2e0c0fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- 172.21.15.151:0/3141505043 >> 172.21.15.151:0/3141505043 conn(0x7f2e0c0f6ee0 msgr2=0x7f2e0c0f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- 172.21.15.151:0/3141505043 shutdown_connections 2024-09-10T18:34:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- 172.21.15.151:0/3141505043 wait complete. 2024-09-10T18:34:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 Processor -- start 2024-09-10T18:34:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- start start 2024-09-10T18:34:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fb740 0x7f2e0c10aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e0c0fca10 0x7f2e0c10b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0fd3c0 0x7f2e0c112080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c112650 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1127c0 con 0x7f2e0c0fb740 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e1163b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c112980 con 0x7f2e0c0fca10 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e0affd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fb740 0x7f2e0c10aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e0affd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fb740 0x7f2e0c10aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55028/0 (socket says 172.21.15.151:55028) 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e0affd640 1 -- 172.21.15.151:0/1989688162 learned_addr learned my addr 172.21.15.151:0/1989688162 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e0b7fe640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0fd3c0 0x7f2e0c112080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.010+0000 7f2e0a7fc640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e0c0fca10 0x7f2e0c10b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e0b7fe640 1 -- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e0c0fca10 msgr2=0x7f2e0c10b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e0b7fe640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e0c0fca10 0x7f2e0c10b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e0b7fe640 1 -- 172.21.15.151:0/1989688162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fb740 msgr2=0x7f2e0c10aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e0b7fe640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fb740 0x7f2e0c10aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e0b7fe640 1 -- 172.21.15.151:0/1989688162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40098b0 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e0b7fe640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0fd3c0 0x7f2e0c112080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0027d0 tx=0x7f2dfc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e037fe640 1 -- 172.21.15.151:0/1989688162 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc007a20 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c112c60 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e037fe640 1 -- 172.21.15.151:0/1989688162 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f2dfc00f040 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c113100 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e037fe640 1 -- 172.21.15.151:0/1989688162 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc009340 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.014+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dec005040 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.018+0000 7f2e037fe640 1 -- 172.21.15.151:0/1989688162 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 90746+0+0 (secure 0 0 0) 0x7f2dfc009730 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.018+0000 7f2e037fe640 1 --2- 172.21.15.151:0/1989688162 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2df8071020 0x7f2df80734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.018+0000 7f2e037fe640 1 -- 172.21.15.151:0/1989688162 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f2dfc08e3f0 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.018+0000 7f2e0affd640 1 --2- 172.21.15.151:0/1989688162 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2df8071020 0x7f2df80734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:36.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.018+0000 7f2e0affd640 1 --2- 172.21.15.151:0/1989688162 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2df8071020 0x7f2df80734d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e040041d0 tx=0x7f2e0400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:36.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.026+0000 7f2e037fe640 1 -- 172.21.15.151:0/1989688162 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f2dfc05b3b0 con 0x7f2e0c0fd3c0 2024-09-10T18:34:36.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.206+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dec000cc0 con 0x7f2df8071020 2024-09-10T18:34:36.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.222+0000 7f2e037fe640 1 -- 172.21.15.151:0/1989688162 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7f2dec000cc0 con 0x7f2df8071020 2024-09-10T18:34:36.237 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled iscsi.datapool update... 2024-09-10T18:34:36.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2df8071020 msgr2=0x7f2df80734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:36.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 --2- 172.21.15.151:0/1989688162 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2df8071020 0x7f2df80734d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e040041d0 tx=0x7f2e0400b040 comp rx=0 tx=0).stop 2024-09-10T18:34:36.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0fd3c0 msgr2=0x7f2e0c112080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:36.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0fd3c0 0x7f2e0c112080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0027d0 tx=0x7f2dfc009f70 comp rx=0 tx=0).stop 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 shutdown_connections 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 --2- 172.21.15.151:0/1989688162 >> v2:172.21.15.100:6800/3986579393 conn(0x7f2df8071020 0x7f2df80734d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e0c0fd3c0 0x7f2e0c112080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e0c0fca10 0x7f2e0c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 --2- 172.21.15.151:0/1989688162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e0c0fb740 0x7f2e0c10aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 >> 172.21.15.151:0/1989688162 conn(0x7f2e0c0f6ee0 msgr2=0x7f2e0c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 shutdown_connections 2024-09-10T18:34:36.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:36.226+0000 7f2e1163b640 1 -- 172.21.15.151:0/1989688162 wait complete. 2024-09-10T18:34:37.022 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T18:34:37.022 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:34:37.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:34:37.039 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:34:37.039 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:34:37.059 DEBUG:teuthology.orchestra.run.smithi151:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T18:34:37.061 INFO:tasks.cephadm:Adding prometheus.a on smithi151 2024-09-10T18:34:37.061 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi151=a' 2024-09-10T18:34:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: cluster 2024-09-10T18:34:35.300116+0000 mgr.y (mgr.14150) 377 : cluster [DBG] pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 244 B/s wr, 0 op/s 2024-09-10T18:34:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.237452+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.239045+0000 mon.a (mon.0) 688 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.615198+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.616609+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.626277+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.630515+0000 mon.a (mon.0) 692 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.637643+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[18597]: audit 2024-09-10T18:34:36.650450+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: cluster 2024-09-10T18:34:35.300116+0000 mgr.y (mgr.14150) 377 : cluster [DBG] pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 244 B/s wr, 0 op/s 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.237452+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.239045+0000 mon.a (mon.0) 688 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.615198+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.616609+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.626277+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.630515+0000 mon.a (mon.0) 692 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:34:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.637643+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:34:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:36 smithi100 bash[31607]: audit 2024-09-10T18:34:36.650450+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: cluster 2024-09-10T18:34:35.300116+0000 mgr.y (mgr.14150) 377 : cluster [DBG] pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 244 B/s wr, 0 op/s 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.237452+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.239045+0000 mon.a (mon.0) 688 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.615198+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.616609+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.626277+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.630515+0000 mon.a (mon.0) 692 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.637643+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:34:37.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:36 smithi151 bash[22565]: audit 2024-09-10T18:34:36.650450+0000 mon.a (mon.0) 694 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:38.123 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:34:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:37 smithi100 bash[18597]: audit 2024-09-10T18:34:36.224044+0000 mgr.y (mgr.14150) 378 : audit [DBG] from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:37 smithi100 bash[18597]: cephadm 2024-09-10T18:34:36.229457+0000 mgr.y (mgr.14150) 379 : cephadm [INF] Saving service iscsi.datapool spec with placement smithi151=iscsi.a;count:1 2024-09-10T18:34:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:37 smithi100 bash[18597]: cephadm 2024-09-10T18:34:36.651914+0000 mgr.y (mgr.14150) 380 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi151 2024-09-10T18:34:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:37 smithi100 bash[31607]: audit 2024-09-10T18:34:36.224044+0000 mgr.y (mgr.14150) 378 : audit [DBG] from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:37 smithi100 bash[31607]: cephadm 2024-09-10T18:34:36.229457+0000 mgr.y (mgr.14150) 379 : cephadm [INF] Saving service iscsi.datapool spec with placement smithi151=iscsi.a;count:1 2024-09-10T18:34:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:37 smithi100 bash[31607]: cephadm 2024-09-10T18:34:36.651914+0000 mgr.y (mgr.14150) 380 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi151 2024-09-10T18:34:38.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:37 smithi151 bash[22565]: audit 2024-09-10T18:34:36.224044+0000 mgr.y (mgr.14150) 378 : audit [DBG] from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:38.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:37 smithi151 bash[22565]: cephadm 2024-09-10T18:34:36.229457+0000 mgr.y (mgr.14150) 379 : cephadm [INF] Saving service iscsi.datapool spec with placement smithi151=iscsi.a;count:1 2024-09-10T18:34:38.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:37 smithi151 bash[22565]: cephadm 2024-09-10T18:34:36.651914+0000 mgr.y (mgr.14150) 380 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi151 2024-09-10T18:34:39.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:38 smithi151 bash[22565]: cluster 2024-09-10T18:34:37.300989+0000 mgr.y (mgr.14150) 381 : cluster [DBG] pgmap v363: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-09-10T18:34:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:38 smithi100 bash[18597]: cluster 2024-09-10T18:34:37.300989+0000 mgr.y (mgr.14150) 381 : cluster [DBG] pgmap v363: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-09-10T18:34:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:38 smithi100 bash[31607]: cluster 2024-09-10T18:34:37.300989+0000 mgr.y (mgr.14150) 381 : cluster [DBG] pgmap v363: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-09-10T18:34:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:40 smithi151 bash[22565]: cluster 2024-09-10T18:34:39.302083+0000 mgr.y (mgr.14150) 382 : cluster [DBG] pgmap v364: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-09-10T18:34:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:40 smithi151 bash[22565]: audit 2024-09-10T18:34:39.846265+0000 mon.a (mon.0) 695 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:40 smithi151 bash[22565]: audit 2024-09-10T18:34:39.855308+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:40 smithi151 bash[22565]: audit 2024-09-10T18:34:39.862186+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:40 smithi151 bash[22565]: cephadm 2024-09-10T18:34:39.863096+0000 mgr.y (mgr.14150) 383 : cephadm [INF] Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:34:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:40 smithi151 bash[22565]: audit 2024-09-10T18:34:39.881970+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:40 smithi151 bash[22565]: audit 2024-09-10T18:34:39.924252+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[18597]: cluster 2024-09-10T18:34:39.302083+0000 mgr.y (mgr.14150) 382 : cluster [DBG] pgmap v364: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-09-10T18:34:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[18597]: audit 2024-09-10T18:34:39.846265+0000 mon.a (mon.0) 695 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[18597]: audit 2024-09-10T18:34:39.855308+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[18597]: audit 2024-09-10T18:34:39.862186+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[18597]: cephadm 2024-09-10T18:34:39.863096+0000 mgr.y (mgr.14150) 383 : cephadm [INF] Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:34:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[18597]: audit 2024-09-10T18:34:39.881970+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[18597]: audit 2024-09-10T18:34:39.924252+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[31607]: cluster 2024-09-10T18:34:39.302083+0000 mgr.y (mgr.14150) 382 : cluster [DBG] pgmap v364: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-09-10T18:34:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[31607]: audit 2024-09-10T18:34:39.846265+0000 mon.a (mon.0) 695 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[31607]: audit 2024-09-10T18:34:39.855308+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[31607]: audit 2024-09-10T18:34:39.862186+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[31607]: cephadm 2024-09-10T18:34:39.863096+0000 mgr.y (mgr.14150) 383 : cephadm [INF] Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:34:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[31607]: audit 2024-09-10T18:34:39.881970+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:40 smithi100 bash[31607]: audit 2024-09-10T18:34:39.924252+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:41.675 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:41 smithi151 bash[54122]: log file path now is '/var/log/tcmu-runner.log' 2024-09-10T18:34:41.809 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:42.128 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:41 smithi151 bash[22565]: audit 2024-09-10T18:34:40.921761+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:42.128 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:41 smithi151 bash[54154]: debug Started the configuration object watcher 2024-09-10T18:34:42.129 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:41 smithi151 bash[54154]: debug Checking for config object changes every 1s 2024-09-10T18:34:42.129 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:41 smithi151 bash[54154]: debug Processing osd blocklist entries for this node 2024-09-10T18:34:42.179 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: debug Reading the configuration object to update local LIO configuration 2024-09-10T18:34:42.179 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: debug Configuration does not have an entry for this host(smithi151.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T18:34:42.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:41 smithi100 bash[31607]: audit 2024-09-10T18:34:40.921761+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:42.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:41 smithi100 bash[18597]: audit 2024-09-10T18:34:40.921761+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:42.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: * Environment: production 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: Use a production WSGI server instead. 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: * Debug mode: off 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: debug * Running on all addresses. 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: * Running on all addresses. 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: debug * Running on http://[fe80::ec4:7aff:fe88:6f63]:5000/ (Press CTRL+C to quit) 2024-09-10T18:34:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[54154]: * Running on http://[fe80::ec4:7aff:fe88:6f63]:5000/ (Press CTRL+C to quit) 2024-09-10T18:34:43.225 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[22565]: cluster 2024-09-10T18:34:41.303121+0000 mgr.y (mgr.14150) 384 : cluster [DBG] pgmap v365: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:43.226 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[22565]: cluster 2024-09-10T18:34:41.946312+0000 mon.a (mon.0) 701 : cluster [DBG] mgrmap e16: y(active, since 10m), standbys: x 2024-09-10T18:34:43.226 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:42 smithi151 bash[22565]: audit 2024-09-10T18:34:42.096316+0000 mon.b (mon.1) 29 : audit [DBG] from='client.? 172.21.15.151:0/3264113130' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:34:43.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:42 smithi100 bash[31607]: cluster 2024-09-10T18:34:41.303121+0000 mgr.y (mgr.14150) 384 : cluster [DBG] pgmap v365: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:43.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:42 smithi100 bash[31607]: cluster 2024-09-10T18:34:41.946312+0000 mon.a (mon.0) 701 : cluster [DBG] mgrmap e16: y(active, since 10m), standbys: x 2024-09-10T18:34:43.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:42 smithi100 bash[31607]: audit 2024-09-10T18:34:42.096316+0000 mon.b (mon.1) 29 : audit [DBG] from='client.? 172.21.15.151:0/3264113130' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:34:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:42 smithi100 bash[18597]: cluster 2024-09-10T18:34:41.303121+0000 mgr.y (mgr.14150) 384 : cluster [DBG] pgmap v365: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:42 smithi100 bash[18597]: cluster 2024-09-10T18:34:41.946312+0000 mon.a (mon.0) 701 : cluster [DBG] mgrmap e16: y(active, since 10m), standbys: x 2024-09-10T18:34:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:42 smithi100 bash[18597]: audit 2024-09-10T18:34:42.096316+0000 mon.b (mon.1) 29 : audit [DBG] from='client.? 172.21.15.151:0/3264113130' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:34:45.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:44 smithi151 bash[22565]: cluster 2024-09-10T18:34:43.304221+0000 mgr.y (mgr.14150) 385 : cluster [DBG] pgmap v366: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:44 smithi100 bash[18597]: cluster 2024-09-10T18:34:43.304221+0000 mgr.y (mgr.14150) 385 : cluster [DBG] pgmap v366: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:45.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:44 smithi100 bash[31607]: cluster 2024-09-10T18:34:43.304221+0000 mgr.y (mgr.14150) 385 : cluster [DBG] pgmap v366: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 -- 172.21.15.151:0/1571982405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 msgr2=0x7fee6c106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 --2- 172.21.15.151:0/1571982405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c106f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee64004950 tx=0x7fee64009da0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 -- 172.21.15.151:0/1571982405 shutdown_connections 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 --2- 172.21.15.151:0/1571982405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c106f10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 --2- 172.21.15.151:0/1571982405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c0ffcc0 0x7fee6c100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 --2- 172.21.15.151:0/1571982405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee6c0fe9f0 0x7fee6c0fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 -- 172.21.15.151:0/1571982405 >> 172.21.15.151:0/1571982405 conn(0x7fee6c0fa200 msgr2=0x7fee6c0fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:45.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 -- 172.21.15.151:0/1571982405 shutdown_connections 2024-09-10T18:34:45.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.306+0000 7fee712fe640 1 -- 172.21.15.151:0/1571982405 wait complete. 2024-09-10T18:34:45.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 Processor -- start 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 -- start start 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee6c0fe9f0 0x7fee6c10ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c0ffcc0 0x7fee6c10f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c116820 con 0x7fee6c0ffcc0 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c116990 con 0x7fee6c100670 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c116b50 con 0x7fee6c0fe9f0 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c116250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53578/0 (socket says 172.21.15.151:53578) 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 -- 172.21.15.151:0/2322109039 learned_addr learned my addr 172.21.15.151:0/2322109039 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:34:45.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6affd640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee6c0fe9f0 0x7fee6c10ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:45.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6a7fc640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c0ffcc0 0x7fee6c10f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:45.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 -- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee6c0fe9f0 msgr2=0x7fee6c10ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:45.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee6c0fe9f0 0x7fee6c10ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 -- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c0ffcc0 msgr2=0x7fee6c10f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c0ffcc0 0x7fee6c10f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 -- 172.21.15.151:0/2322109039 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64005040 con 0x7fee6c100670 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee6b7fe640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c116250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee64004920 tx=0x7fee64009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee537fe640 1 -- 172.21.15.151:0/2322109039 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6402bd00 con 0x7fee6c100670 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee537fe640 1 -- 172.21.15.151:0/2322109039 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fee6402be60 con 0x7fee6c100670 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c116dd0 con 0x7fee6c100670 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee537fe640 1 -- 172.21.15.151:0/2322109039 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6403c750 con 0x7fee6c100670 2024-09-10T18:34:45.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c1172f0 con 0x7fee6c100670 2024-09-10T18:34:45.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.310+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c04bf60 con 0x7fee6c100670 2024-09-10T18:34:45.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.314+0000 7fee537fe640 1 -- 172.21.15.151:0/2322109039 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) v1 ==== 90797+0+0 (secure 0 0 0) 0x7fee64030020 con 0x7fee6c100670 2024-09-10T18:34:45.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.314+0000 7fee537fe640 1 --2- 172.21.15.151:0/2322109039 >> v2:172.21.15.100:6800/3986579393 conn(0x7fee5406ee90 0x7fee54071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:45.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.314+0000 7fee537fe640 1 -- 172.21.15.151:0/2322109039 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5938+0+0 (secure 0 0 0) 0x7fee640ae820 con 0x7fee6c100670 2024-09-10T18:34:45.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.314+0000 7fee6affd640 1 --2- 172.21.15.151:0/2322109039 >> v2:172.21.15.100:6800/3986579393 conn(0x7fee5406ee90 0x7fee54071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:45.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.314+0000 7fee6affd640 1 --2- 172.21.15.151:0/2322109039 >> v2:172.21.15.100:6800/3986579393 conn(0x7fee5406ee90 0x7fee54071340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fee6c065240 tx=0x7fee6000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:45.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.318+0000 7fee537fe640 1 -- 172.21.15.151:0/2322109039 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fee6407b920 con 0x7fee6c100670 2024-09-10T18:34:45.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.502+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fee6c05db50 con 0x7fee5406ee90 2024-09-10T18:34:45.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.518+0000 7fee537fe640 1 -- 172.21.15.151:0/2322109039 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fee6c05db50 con 0x7fee5406ee90 2024-09-10T18:34:45.532 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled prometheus update... 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 >> v2:172.21.15.100:6800/3986579393 conn(0x7fee5406ee90 msgr2=0x7fee54071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 --2- 172.21.15.151:0/2322109039 >> v2:172.21.15.100:6800/3986579393 conn(0x7fee5406ee90 0x7fee54071340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fee6c065240 tx=0x7fee6000b040 comp rx=0 tx=0).stop 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 msgr2=0x7fee6c116250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c116250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fee64004920 tx=0x7fee64009da0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 shutdown_connections 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 --2- 172.21.15.151:0/2322109039 >> v2:172.21.15.100:6800/3986579393 conn(0x7fee5406ee90 0x7fee54071340 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee6c100670 0x7fee6c116250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee6c0ffcc0 0x7fee6c10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 --2- 172.21.15.151:0/2322109039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee6c0fe9f0 0x7fee6c10ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:45.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 >> 172.21.15.151:0/2322109039 conn(0x7fee6c0fa200 msgr2=0x7fee6c0fbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:45.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 shutdown_connections 2024-09-10T18:34:45.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:45.522+0000 7fee712fe640 1 -- 172.21.15.151:0/2322109039 wait complete. 2024-09-10T18:34:46.350 DEBUG:teuthology.orchestra.run.smithi151:prometheus.a> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T18:34:46.353 INFO:tasks.cephadm:Adding node-exporter.a on smithi100 2024-09-10T18:34:46.353 INFO:tasks.cephadm:Adding node-exporter.b on smithi151 2024-09-10T18:34:46.353 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi100=a;smithi151=b' 2024-09-10T18:34:46.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.290868+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.302312+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.304058+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: cluster 2024-09-10T18:34:45.305214+0000 mgr.y (mgr.14150) 386 : cluster [DBG] pgmap v367: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 368 B/s rd, 92 B/s wr, 1 op/s 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.306396+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.314850+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.371657+0000 mon.a (mon.0) 707 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.372413+0000 mgr.y (mgr.14150) 387 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: cephadm 2024-09-10T18:34:45.374429+0000 mgr.y (mgr.14150) 388 : cephadm [INF] Adding iSCSI gateway http://:@172.21.15.151:5000 to Dashboard 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.374987+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.375502+0000 mgr.y (mgr.14150) 389 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.387434+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.397246+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi151"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.398363+0000 mgr.y (mgr.14150) 390 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi151"}]: dispatch 2024-09-10T18:34:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.408824+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.411188+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.413645+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.414883+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.425446+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.290868+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.302312+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.304058+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: cluster 2024-09-10T18:34:45.305214+0000 mgr.y (mgr.14150) 386 : cluster [DBG] pgmap v367: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 368 B/s rd, 92 B/s wr, 1 op/s 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.306396+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.314850+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.371657+0000 mon.a (mon.0) 707 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.372413+0000 mgr.y (mgr.14150) 387 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: cephadm 2024-09-10T18:34:45.374429+0000 mgr.y (mgr.14150) 388 : cephadm [INF] Adding iSCSI gateway http://:@172.21.15.151:5000 to Dashboard 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.374987+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.375502+0000 mgr.y (mgr.14150) 389 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.387434+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.397246+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi151"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.398363+0000 mgr.y (mgr.14150) 390 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi151"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.408824+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.411188+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.413645+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.414883+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.425446+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.519682+0000 mgr.y (mgr.14150) 391 : audit [DBG] from='client.24496 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: cephadm 2024-09-10T18:34:45.522619+0000 mgr.y (mgr.14150) 392 : cephadm [INF] Saving service prometheus spec with placement smithi151=a;count:1 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.532071+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.533804+0000 mon.a (mon.0) 717 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.537122+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.538826+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: audit 2024-09-10T18:34:45.550573+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[18597]: cephadm 2024-09-10T18:34:45.570125+0000 mgr.y (mgr.14150) 393 : cephadm [INF] Deploying daemon prometheus.a on smithi151 2024-09-10T18:34:46.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.519682+0000 mgr.y (mgr.14150) 391 : audit [DBG] from='client.24496 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:46.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: cephadm 2024-09-10T18:34:45.522619+0000 mgr.y (mgr.14150) 392 : cephadm [INF] Saving service prometheus spec with placement smithi151=a;count:1 2024-09-10T18:34:46.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.532071+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.533804+0000 mon.a (mon.0) 717 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:46.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.537122+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.538826+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: audit 2024-09-10T18:34:45.550573+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:46 smithi100 bash[31607]: cephadm 2024-09-10T18:34:45.570125+0000 mgr.y (mgr.14150) 393 : cephadm [INF] Deploying daemon prometheus.a on smithi151 2024-09-10T18:34:46.637 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:34:46.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.290868+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.302312+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.304058+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: cluster 2024-09-10T18:34:45.305214+0000 mgr.y (mgr.14150) 386 : cluster [DBG] pgmap v367: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 368 B/s rd, 92 B/s wr, 1 op/s 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.306396+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.314850+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.371657+0000 mon.a (mon.0) 707 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.372413+0000 mgr.y (mgr.14150) 387 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: cephadm 2024-09-10T18:34:45.374429+0000 mgr.y (mgr.14150) 388 : cephadm [INF] Adding iSCSI gateway http://:@172.21.15.151:5000 to Dashboard 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.374987+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.375502+0000 mgr.y (mgr.14150) 389 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.387434+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.397246+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi151"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.398363+0000 mgr.y (mgr.14150) 390 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi151"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.408824+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.411188+0000 mon.a (mon.0) 712 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:46.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.413645+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.414883+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.425446+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.519682+0000 mgr.y (mgr.14150) 391 : audit [DBG] from='client.24496 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: cephadm 2024-09-10T18:34:45.522619+0000 mgr.y (mgr.14150) 392 : cephadm [INF] Saving service prometheus spec with placement smithi151=a;count:1 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.532071+0000 mon.a (mon.0) 716 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.533804+0000 mon.a (mon.0) 717 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.537122+0000 mon.a (mon.0) 718 : audit [DBG] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.538826+0000 mon.a (mon.0) 719 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: audit 2024-09-10T18:34:45.550573+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:34:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:46 smithi151 bash[22565]: cephadm 2024-09-10T18:34:45.570125+0000 mgr.y (mgr.14150) 393 : cephadm [INF] Deploying daemon prometheus.a on smithi151 2024-09-10T18:34:48.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:48 smithi151 bash[22565]: cluster 2024-09-10T18:34:47.306110+0000 mgr.y (mgr.14150) 394 : cluster [DBG] pgmap v368: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:48.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:48 smithi100 bash[18597]: cluster 2024-09-10T18:34:47.306110+0000 mgr.y (mgr.14150) 394 : cluster [DBG] pgmap v368: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:48.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:48 smithi100 bash[31607]: cluster 2024-09-10T18:34:47.306110+0000 mgr.y (mgr.14150) 394 : cluster [DBG] pgmap v368: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:50.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:50 smithi151 bash[22565]: cluster 2024-09-10T18:34:49.307228+0000 mgr.y (mgr.14150) 395 : cluster [DBG] pgmap v369: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:50 smithi100 bash[18597]: cluster 2024-09-10T18:34:49.307228+0000 mgr.y (mgr.14150) 395 : cluster [DBG] pgmap v369: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:50.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:50 smithi100 bash[31607]: cluster 2024-09-10T18:34:49.307228+0000 mgr.y (mgr.14150) 395 : cluster [DBG] pgmap v369: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 341 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:51.152 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:52.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:34:51 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:34:52.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:52 smithi151 bash[22565]: cluster 2024-09-10T18:34:51.308296+0000 mgr.y (mgr.14150) 396 : cluster [DBG] pgmap v370: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:52.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:52 smithi151 bash[22565]: audit 2024-09-10T18:34:51.841412+0000 mgr.y (mgr.14150) 397 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:34:52.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:52 smithi100 bash[18597]: cluster 2024-09-10T18:34:51.308296+0000 mgr.y (mgr.14150) 396 : cluster [DBG] pgmap v370: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:52.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:52 smithi100 bash[18597]: audit 2024-09-10T18:34:51.841412+0000 mgr.y (mgr.14150) 397 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:34:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:52 smithi100 bash[31607]: cluster 2024-09-10T18:34:51.308296+0000 mgr.y (mgr.14150) 396 : cluster [DBG] pgmap v370: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:52 smithi100 bash[31607]: audit 2024-09-10T18:34:51.841412+0000 mgr.y (mgr.14150) 397 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:34:54.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:54 smithi151 bash[22565]: cluster 2024-09-10T18:34:53.309400+0000 mgr.y (mgr.14150) 398 : cluster [DBG] pgmap v371: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:54 smithi100 bash[18597]: cluster 2024-09-10T18:34:53.309400+0000 mgr.y (mgr.14150) 398 : cluster [DBG] pgmap v371: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:54 smithi100 bash[31607]: cluster 2024-09-10T18:34:53.309400+0000 mgr.y (mgr.14150) 398 : cluster [DBG] pgmap v371: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:56.629 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:56 smithi151 bash[22565]: cluster 2024-09-10T18:34:55.310482+0000 mgr.y (mgr.14150) 399 : cluster [DBG] pgmap v372: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:56 smithi100 bash[18597]: cluster 2024-09-10T18:34:55.310482+0000 mgr.y (mgr.14150) 399 : cluster [DBG] pgmap v372: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:56.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:56 smithi100 bash[31607]: cluster 2024-09-10T18:34:55.310482+0000 mgr.y (mgr.14150) 399 : cluster [DBG] pgmap v372: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:34:58.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:34:58 smithi151 bash[22565]: cluster 2024-09-10T18:34:57.311265+0000 mgr.y (mgr.14150) 400 : cluster [DBG] pgmap v373: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:34:58.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:34:58 smithi100 bash[18597]: cluster 2024-09-10T18:34:57.311265+0000 mgr.y (mgr.14150) 400 : cluster [DBG] pgmap v373: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:34:58.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:34:58 smithi100 bash[31607]: cluster 2024-09-10T18:34:57.311265+0000 mgr.y (mgr.14150) 400 : cluster [DBG] pgmap v373: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:34:58.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 -- 172.21.15.151:0/3193208846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04680fea20 msgr2=0x7f04680fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 --2- 172.21.15.151:0/3193208846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04680fea20 0x7f04680fee30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0458004950 tx=0x7f0458009da0 comp rx=0 tx=0).stop 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 -- 172.21.15.151:0/3193208846 shutdown_connections 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 --2- 172.21.15.151:0/3193208846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04681006a0 0x7f0468106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 --2- 172.21.15.151:0/3193208846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04680ffcf0 0x7f0468100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 --2- 172.21.15.151:0/3193208846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04680fea20 0x7f04680fee30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 -- 172.21.15.151:0/3193208846 >> 172.21.15.151:0/3193208846 conn(0x7f04680fa200 msgr2=0x7f04680fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 -- 172.21.15.151:0/3193208846 shutdown_connections 2024-09-10T18:34:58.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.774+0000 7f046f3b4640 1 -- 172.21.15.151:0/3193208846 wait complete. 2024-09-10T18:34:58.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 Processor -- start 2024-09-10T18:34:58.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 -- start start 2024-09-10T18:34:58.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04680fea20 0x7f046810ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:58.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04680ffcf0 0x7f046810f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04681006a0 0x7f04681164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468116a70 con 0x7f04681006a0 2024-09-10T18:34:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0468116be0 con 0x7f04680fea20 2024-09-10T18:34:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0468116da0 con 0x7f04680ffcf0 2024-09-10T18:34:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04680fea20 0x7f046810ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04680fea20 0x7f046810ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51208/0 (socket says 172.21.15.151:51208) 2024-09-10T18:34:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 -- 172.21.15.151:0/2787404645 learned_addr learned my addr 172.21.15.151:0/2787404645 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046c94f640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04680ffcf0 0x7f046810f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d951640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04681006a0 0x7f04681164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 -- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04680ffcf0 msgr2=0x7f046810f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04680ffcf0 0x7f046810f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 -- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04681006a0 msgr2=0x7f04681164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04681006a0 0x7f04681164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 -- 172.21.15.151:0/2787404645 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458005040 con 0x7f04680fea20 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d951640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04681006a0 0x7f04681164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:34:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046d150640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04680fea20 0x7f046810ef00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04580049b0 tx=0x7f0458009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:58.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f045dffb640 1 -- 172.21.15.151:0/2787404645 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045802ab70 con 0x7f04680fea20 2024-09-10T18:34:58.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f045dffb640 1 -- 172.21.15.151:0/2787404645 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f045802acd0 con 0x7f04680fea20 2024-09-10T18:34:58.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 -- 172.21.15.151:0/2787404645 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0468117020 con 0x7f04680fea20 2024-09-10T18:34:58.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f045dffb640 1 -- 172.21.15.151:0/2787404645 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0458032ec0 con 0x7f04680fea20 2024-09-10T18:34:58.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f046f3b4640 1 -- 172.21.15.151:0/2787404645 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04681174f0 con 0x7f04680fea20 2024-09-10T18:34:58.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.778+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0448005010 con 0x7f04680fea20 2024-09-10T18:34:58.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.782+0000 7f045dffb640 1 -- 172.21.15.151:0/2787404645 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) v1 ==== 90797+0+0 (secure 0 0 0) 0x7f0458030020 con 0x7f04680fea20 2024-09-10T18:34:58.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.782+0000 7f045dffb640 1 --2- 172.21.15.151:0/2787404645 >> v2:172.21.15.100:6800/3986579393 conn(0x7f045406ee90 0x7f0454071340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:34:58.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.782+0000 7f045dffb640 1 -- 172.21.15.151:0/2787404645 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5938+0+0 (secure 0 0 0) 0x7f04580ae0f0 con 0x7f04680fea20 2024-09-10T18:34:58.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.782+0000 7f046c94f640 1 --2- 172.21.15.151:0/2787404645 >> v2:172.21.15.100:6800/3986579393 conn(0x7f045406ee90 0x7f0454071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:34:58.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.782+0000 7f046c94f640 1 --2- 172.21.15.151:0/2787404645 >> v2:172.21.15.100:6800/3986579393 conn(0x7f045406ee90 0x7f0454071340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0468110400 tx=0x7f046400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:34:58.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.786+0000 7f045dffb640 1 -- 172.21.15.151:0/2787404645 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f045807b0c0 con 0x7f04680fea20 2024-09-10T18:34:59.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.986+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 --> v2:172.21.15.100:6800/3986579393 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi151=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f0448000bc0 con 0x7f045406ee90 2024-09-10T18:34:59.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:58.998+0000 7f045dffb640 1 -- 172.21.15.151:0/2787404645 <== mgr.14150 v2:172.21.15.100:6800/3986579393 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f0448000bc0 con 0x7f045406ee90 2024-09-10T18:34:59.018 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled node-exporter update... 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 >> v2:172.21.15.100:6800/3986579393 conn(0x7f045406ee90 msgr2=0x7f0454071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 --2- 172.21.15.151:0/2787404645 >> v2:172.21.15.100:6800/3986579393 conn(0x7f045406ee90 0x7f0454071340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0468110400 tx=0x7f046400b040 comp rx=0 tx=0).stop 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04680fea20 msgr2=0x7f046810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04680fea20 0x7f046810ef00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f04580049b0 tx=0x7f0458009da0 comp rx=0 tx=0).stop 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 shutdown_connections 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 --2- 172.21.15.151:0/2787404645 >> v2:172.21.15.100:6800/3986579393 conn(0x7f045406ee90 0x7f0454071340 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04681006a0 0x7f04681164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04680ffcf0 0x7f046810f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:59.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 --2- 172.21.15.151:0/2787404645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04680fea20 0x7f046810ef00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:34:59.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 >> 172.21.15.151:0/2787404645 conn(0x7f04680fa200 msgr2=0x7f04680fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:59.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 shutdown_connections 2024-09-10T18:34:59.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:34:59.002+0000 7f04537fe640 1 -- 172.21.15.151:0/2787404645 wait complete. 2024-09-10T18:35:00.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:00 smithi151 bash[22565]: audit 2024-09-10T18:34:59.007919+0000 mgr.y (mgr.14150) 401 : audit [DBG] from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:00.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:00 smithi151 bash[22565]: cephadm 2024-09-10T18:34:59.011042+0000 mgr.y (mgr.14150) 402 : cephadm [INF] Saving service node-exporter spec with placement smithi100=a;smithi151=b;count:2 2024-09-10T18:35:00.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:00 smithi151 bash[22565]: audit 2024-09-10T18:34:59.018245+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:00.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:00 smithi151 bash[22565]: cluster 2024-09-10T18:34:59.312346+0000 mgr.y (mgr.14150) 403 : cluster [DBG] pgmap v374: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:00.204 DEBUG:teuthology.orchestra.run.smithi100:node-exporter.a> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T18:35:00.207 DEBUG:teuthology.orchestra.run.smithi151:node-exporter.b> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T18:35:00.210 INFO:tasks.cephadm:Adding alertmanager.a on smithi100 2024-09-10T18:35:00.210 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi100=a' 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[18597]: audit 2024-09-10T18:34:59.007919+0000 mgr.y (mgr.14150) 401 : audit [DBG] from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[18597]: cephadm 2024-09-10T18:34:59.011042+0000 mgr.y (mgr.14150) 402 : cephadm [INF] Saving service node-exporter spec with placement smithi100=a;smithi151=b;count:2 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[18597]: audit 2024-09-10T18:34:59.018245+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[18597]: cluster 2024-09-10T18:34:59.312346+0000 mgr.y (mgr.14150) 403 : cluster [DBG] pgmap v374: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[31607]: audit 2024-09-10T18:34:59.007919+0000 mgr.y (mgr.14150) 401 : audit [DBG] from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[31607]: cephadm 2024-09-10T18:34:59.011042+0000 mgr.y (mgr.14150) 402 : cephadm [INF] Saving service node-exporter spec with placement smithi100=a;smithi151=b;count:2 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[31607]: audit 2024-09-10T18:34:59.018245+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:00 smithi100 bash[31607]: cluster 2024-09-10T18:34:59.312346+0000 mgr.y (mgr.14150) 403 : cluster [DBG] pgmap v374: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:00.333 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:35:00.393 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:35:02.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:35:01 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:35:02.590 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:02 smithi151 bash[22565]: cluster 2024-09-10T18:35:01.313340+0000 mgr.y (mgr.14150) 404 : cluster [DBG] pgmap v375: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:02.590 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:02 smithi151 bash[22565]: audit 2024-09-10T18:35:01.852749+0000 mgr.y (mgr.14150) 405 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:02 smithi100 bash[18597]: cluster 2024-09-10T18:35:01.313340+0000 mgr.y (mgr.14150) 404 : cluster [DBG] pgmap v375: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:02 smithi100 bash[18597]: audit 2024-09-10T18:35:01.852749+0000 mgr.y (mgr.14150) 405 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:02.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:02 smithi100 bash[31607]: cluster 2024-09-10T18:35:01.313340+0000 mgr.y (mgr.14150) 404 : cluster [DBG] pgmap v375: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:02.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:02 smithi100 bash[31607]: audit 2024-09-10T18:35:01.852749+0000 mgr.y (mgr.14150) 405 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:02.925 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:02 smithi151 systemd[1]: Started Ceph prometheus.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:35:04.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:03 smithi151 bash[22565]: audit 2024-09-10T18:35:02.864084+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.169 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:03 smithi151 bash[22565]: audit 2024-09-10T18:35:02.873802+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.169 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:03 smithi151 bash[22565]: audit 2024-09-10T18:35:02.880547+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.169 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:03 smithi151 bash[22565]: audit 2024-09-10T18:35:02.886060+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:35:04.169 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:03 smithi151 bash[23560]: ignoring --setuser ceph since I am not root 2024-09-10T18:35:04.169 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:03 smithi151 bash[23560]: ignoring --setgroup ceph since I am not root 2024-09-10T18:35:04.169 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:03 smithi151 bash[23560]: debug 2024-09-10T18:35:03.926+0000 7f917935e640 1 -- 172.21.15.151:0/3875885921 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8780d4340 con 0x55f8780e0c00 2024-09-10T18:35:04.169 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[23560]: debug 2024-09-10T18:35:04.054+0000 7f917bbb8ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:35:04.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[18863]: ignoring --setuser ceph since I am not root 2024-09-10T18:35:04.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[18863]: ignoring --setgroup ceph since I am not root 2024-09-10T18:35:04.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[18863]: debug 2024-09-10T18:35:03.938+0000 7f188c545640 1 -- 172.21.15.100:0/3626738788 <== mon.2 v2:172.21.15.100:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55776e560340 con 0x55776e56c800 2024-09-10T18:35:04.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[18863]: debug 2024-09-10T18:35:04.058+0000 7f188ed9fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:35:04.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[31607]: audit 2024-09-10T18:35:02.864084+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[31607]: audit 2024-09-10T18:35:02.873802+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[31607]: audit 2024-09-10T18:35:02.880547+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[31607]: audit 2024-09-10T18:35:02.886060+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:35:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[18597]: audit 2024-09-10T18:35:02.864084+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[18597]: audit 2024-09-10T18:35:02.873802+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[18597]: audit 2024-09-10T18:35:02.880547+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' 2024-09-10T18:35:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:03 smithi100 bash[18597]: audit 2024-09-10T18:35:02.886060+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:35:04.426 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.155Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:35:04.426 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.155Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:35:04.426 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.155Z caller=main.go:570 level=info host_details="(Linux 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 smithi151 (none))" 2024-09-10T18:35:04.426 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.155Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:35:04.426 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.155Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:35:04.426 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.157Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.157Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.160Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.160Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.164Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.164Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.934µs 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.164Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.165Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.165Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=36.246µs wal_replay_duration=492.092µs wbl_replay_duration=150ns total_replay_duration=581.161µs 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.167Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.167Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.167Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.191Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=24.108718ms db_storage=905ns remote_storage=1.348µs web_handler=445ns query_engine=671ns scrape=375.82µs scrape_sd=24.299µs notify=932ns notify_sd=1.947µs rules=23.310683ms tracing=8.906µs 2024-09-10T18:35:04.427 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.191Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:35:04.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[55308]: ts=2024-09-10T18:35:04.191Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:35:05.086 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[18597]: audit 2024-09-10T18:35:03.899842+0000 mon.a (mon.0) 726 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:35:05.086 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[18597]: cluster 2024-09-10T18:35:03.899901+0000 mon.a (mon.0) 727 : cluster [DBG] mgrmap e17: y(active, since 10m), standbys: x 2024-09-10T18:35:05.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[31607]: audit 2024-09-10T18:35:03.899842+0000 mon.a (mon.0) 726 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:35:05.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[31607]: cluster 2024-09-10T18:35:03.899901+0000 mon.a (mon.0) 727 : cluster [DBG] mgrmap e17: y(active, since 10m), standbys: x 2024-09-10T18:35:05.087 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[18863]: debug 2024-09-10T18:35:04.798+0000 7f188ed9fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:35:05.087 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[18863]: debug 2024-09-10T18:35:04.862+0000 7f188ed9fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:35:05.087 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:04 smithi100 bash[18863]: debug 2024-09-10T18:35:04.930+0000 7f188ed9fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:35:05.143 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[22565]: audit 2024-09-10T18:35:03.899842+0000 mon.a (mon.0) 726 : audit [INF] from='mgr.14150 172.21.15.100:0/1498605322' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:35:05.143 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[22565]: cluster 2024-09-10T18:35:03.899901+0000 mon.a (mon.0) 727 : cluster [DBG] mgrmap e17: y(active, since 10m), standbys: x 2024-09-10T18:35:05.143 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[23560]: debug 2024-09-10T18:35:04.838+0000 7f917bbb8ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:35:05.143 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[23560]: debug 2024-09-10T18:35:04.906+0000 7f917bbb8ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:35:05.143 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:04 smithi151 bash[23560]: debug 2024-09-10T18:35:04.970+0000 7f917bbb8ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:35:05.143 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:05 smithi151 bash[23560]: debug 2024-09-10T18:35:05.122+0000 7f917bbb8ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:05.396 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:05 smithi100 bash[18863]: debug 2024-09-10T18:35:05.082+0000 7f188ed9fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:05.396 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:05 smithi100 bash[18863]: debug 2024-09-10T18:35:05.330+0000 7f188ed9fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:35:05.425 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:05 smithi151 bash[23560]: debug 2024-09-10T18:35:05.370+0000 7f917bbb8ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:35:05.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:05 smithi100 bash[18863]: debug 2024-09-10T18:35:05.394+0000 7f188ed9fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:05.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:05 smithi100 bash[18863]: debug 2024-09-10T18:35:05.526+0000 7f188ed9fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:35:05.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:05 smithi151 bash[23560]: debug 2024-09-10T18:35:05.430+0000 7f917bbb8ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:05.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:05 smithi151 bash[23560]: debug 2024-09-10T18:35:05.566+0000 7f917bbb8ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:35:06.022 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:35:06.425 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:06 smithi151 bash[23560]: debug 2024-09-10T18:35:06.194+0000 7f917bbb8ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:35:06.425 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:06 smithi151 bash[23560]: debug 2024-09-10T18:35:06.266+0000 7f917bbb8ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:35:06.425 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:06 smithi151 bash[23560]: debug 2024-09-10T18:35:06.338+0000 7f917bbb8ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:35:06.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:06 smithi100 bash[18863]: debug 2024-09-10T18:35:06.146+0000 7f188ed9fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:35:06.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:06 smithi100 bash[18863]: debug 2024-09-10T18:35:06.218+0000 7f188ed9fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:35:06.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:06 smithi100 bash[18863]: debug 2024-09-10T18:35:06.294+0000 7f188ed9fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:35:06.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:06 smithi151 bash[23560]: debug 2024-09-10T18:35:06.622+0000 7f917bbb8ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:35:06.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:06 smithi151 bash[23560]: debug 2024-09-10T18:35:06.758+0000 7f917bbb8ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:35:07.023 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:06 smithi100 bash[18863]: debug 2024-09-10T18:35:06.574+0000 7f188ed9fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:35:07.023 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:06 smithi100 bash[18863]: debug 2024-09-10T18:35:06.710+0000 7f188ed9fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:35:07.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:07 smithi100 bash[18863]: debug 2024-09-10T18:35:07.018+0000 7f188ed9fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:35:07.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:07 smithi100 bash[18863]: debug 2024-09-10T18:35:07.234+0000 7f188ed9fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:07.376 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:07 smithi151 bash[23560]: debug 2024-09-10T18:35:07.066+0000 7f917bbb8ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:35:07.377 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:07 smithi151 bash[23560]: debug 2024-09-10T18:35:07.286+0000 7f917bbb8ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:07.935 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:07 smithi100 bash[18863]: debug 2024-09-10T18:35:07.646+0000 7f188ed9fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:35:07.935 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:07 smithi100 bash[18863]: debug 2024-09-10T18:35:07.710+0000 7f188ed9fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:35:07.935 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:07 smithi100 bash[18863]: debug 2024-09-10T18:35:07.774+0000 7f188ed9fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:35:07.935 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:07 smithi100 bash[18863]: debug 2024-09-10T18:35:07.866+0000 7f188ed9fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:07.935 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:07 smithi100 bash[18863]: debug 2024-09-10T18:35:07.930+0000 7f188ed9fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:35:07.955 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:07 smithi151 bash[23560]: debug 2024-09-10T18:35:07.718+0000 7f917bbb8ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:35:07.955 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:07 smithi151 bash[23560]: debug 2024-09-10T18:35:07.778+0000 7f917bbb8ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:35:07.956 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:07 smithi151 bash[23560]: debug 2024-09-10T18:35:07.842+0000 7f917bbb8ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:35:08.230 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:08 smithi100 bash[18863]: debug 2024-09-10T18:35:08.006+0000 7f188ed9fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:35:08.230 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:08 smithi100 bash[18863]: debug 2024-09-10T18:35:08.150+0000 7f188ed9fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:35:08.230 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:08 smithi100 bash[18863]: debug 2024-09-10T18:35:08.226+0000 7f188ed9fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:35:08.241 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:07 smithi151 bash[23560]: debug 2024-09-10T18:35:07.938+0000 7f917bbb8ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:08.241 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:08 smithi151 bash[23560]: debug 2024-09-10T18:35:08.002+0000 7f917bbb8ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:35:08.241 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:08 smithi151 bash[23560]: debug 2024-09-10T18:35:08.078+0000 7f917bbb8ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:35:08.675 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:08 smithi151 bash[23560]: debug 2024-09-10T18:35:08.222+0000 7f917bbb8ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:35:08.675 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:08 smithi151 bash[23560]: debug 2024-09-10T18:35:08.302+0000 7f917bbb8ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: cluster 2024-09-10T18:35:09.192963+0000 mon.a (mon.0) 728 : cluster [INF] Active manager daemon y restarted 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: cluster 2024-09-10T18:35:09.193925+0000 mon.a (mon.0) 729 : cluster [INF] Activating manager daemon y 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: cluster 2024-09-10T18:35:09.211755+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: cluster 2024-09-10T18:35:09.212698+0000 mon.a (mon.0) 731 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0191173s), standbys: x 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.216765+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.217032+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.217214+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.218423+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:35:09.427 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.218673+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.219004+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.219359+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.219663+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.220022+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.220438+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.220714+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.220974+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.221247+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.221552+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.221744+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:35:09.428 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18863]: debug 2024-09-10T18:35:09.166+0000 7f188ed9fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:35:09.429 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18863]: [10/Sep/2024:18:35:09] ENGINE Bus STARTING 2024-09-10T18:35:09.429 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18863]: CherryPy Checker: 2024-09-10T18:35:09.429 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18863]: The Application mounted at '' has an empty config. 2024-09-10T18:35:09.429 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18863]: [10/Sep/2024:18:35:09] ENGINE Serving on http://:::9283 2024-09-10T18:35:09.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: audit 2024-09-10T18:35:09.222254+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:35:09.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18597]: cluster 2024-09-10T18:35:09.230795+0000 mon.a (mon.0) 732 : cluster [INF] Manager daemon y is now available 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: cluster 2024-09-10T18:35:09.192963+0000 mon.a (mon.0) 728 : cluster [INF] Active manager daemon y restarted 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: cluster 2024-09-10T18:35:09.193925+0000 mon.a (mon.0) 729 : cluster [INF] Activating manager daemon y 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: cluster 2024-09-10T18:35:09.211755+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: cluster 2024-09-10T18:35:09.212698+0000 mon.a (mon.0) 731 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0191173s), standbys: x 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.216765+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.217032+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.217214+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.218423+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.218673+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.219004+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.219359+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:09.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.219663+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.220022+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.220438+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.220714+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.220974+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.221247+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.221552+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.221744+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: audit 2024-09-10T18:35:09.222254+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:35:09.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[31607]: cluster 2024-09-10T18:35:09.230795+0000 mon.a (mon.0) 732 : cluster [INF] Manager daemon y is now available 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 -- 172.21.15.151:0/1450331258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28140fea20 msgr2=0x7f28140fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 --2- 172.21.15.151:0/1450331258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28140fea20 0x7f28140fee30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2810004950 tx=0x7f2810009da0 comp rx=0 tx=0).stop 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 -- 172.21.15.151:0/1450331258 shutdown_connections 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 --2- 172.21.15.151:0/1450331258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28141006a0 0x7f2814106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 --2- 172.21.15.151:0/1450331258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28140ffcf0 0x7f2814100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 --2- 172.21.15.151:0/1450331258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28140fea20 0x7f28140fee30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 -- 172.21.15.151:0/1450331258 >> 172.21.15.151:0/1450331258 conn(0x7f28140fa200 msgr2=0x7f28140fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 -- 172.21.15.151:0/1450331258 shutdown_connections 2024-09-10T18:35:09.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 -- 172.21.15.151:0/1450331258 wait complete. 2024-09-10T18:35:09.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.574+0000 7f281b695640 1 Processor -- start 2024-09-10T18:35:09.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 -- start start 2024-09-10T18:35:09.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28140fea20 0x7f281410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:09.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ffcf0 0x7f281410f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:09.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28141006a0 0x7f28141164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:09.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814116a70 con 0x7f28140fea20 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814116be0 con 0x7f28141006a0 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2814116da0 con 0x7f28140ffcf0 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28141006a0 0x7f28141164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2818c30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ffcf0 0x7f281410f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28141006a0 0x7f28141164a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52492/0 (socket says 172.21.15.151:52492) 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 -- 172.21.15.151:0/1958812932 learned_addr learned my addr 172.21.15.151:0/1958812932 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819431640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28140fea20 0x7f281410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 -- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ffcf0 msgr2=0x7f281410f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ffcf0 0x7f281410f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 -- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28140fea20 msgr2=0x7f281410ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28140fea20 0x7f281410ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:09.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 -- 172.21.15.151:0/1958812932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810005040 con 0x7f28141006a0 2024-09-10T18:35:09.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2818c30640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ffcf0 0x7f281410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:35:09.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2819c32640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28141006a0 0x7f28141164a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f280c00ece0 tx=0x7f280c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:09.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c018070 con 0x7f28141006a0 2024-09-10T18:35:09.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f280c00f040 con 0x7f28141006a0 2024-09-10T18:35:09.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2814117020 con 0x7f28141006a0 2024-09-10T18:35:09.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c007d40 con 0x7f28141006a0 2024-09-10T18:35:09.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.578+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28141175c0 con 0x7f28141006a0 2024-09-10T18:35:09.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.582+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281404bf60 con 0x7f28141006a0 2024-09-10T18:35:09.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.582+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) v1 ==== 90462+0+0 (secure 0 0 0) 0x7f280c0092e0 con 0x7f28141006a0 2024-09-10T18:35:09.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.582+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f280c08df70 con 0x7f28141006a0 2024-09-10T18:35:09.601 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[23560]: debug 2024-09-10T18:35:09.266+0000 7f917bbb8ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:35:09.601 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[23560]: [10/Sep/2024:18:35:09] ENGINE Bus STARTING 2024-09-10T18:35:09.601 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[23560]: CherryPy Checker: 2024-09-10T18:35:09.601 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[23560]: The Application mounted at '' has an empty config. 2024-09-10T18:35:09.603 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[23560]: [10/Sep/2024:18:35:09] ENGINE Serving on http://:::9283 2024-09-10T18:35:09.603 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[23560]: [10/Sep/2024:18:35:09] ENGINE Bus STARTED 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: cluster 2024-09-10T18:35:09.192963+0000 mon.a (mon.0) 728 : cluster [INF] Active manager daemon y restarted 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: cluster 2024-09-10T18:35:09.193925+0000 mon.a (mon.0) 729 : cluster [INF] Activating manager daemon y 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: cluster 2024-09-10T18:35:09.211755+0000 mon.a (mon.0) 730 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: cluster 2024-09-10T18:35:09.212698+0000 mon.a (mon.0) 731 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0191173s), standbys: x 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.216765+0000 mon.c (mon.2) 17 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.217032+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.217214+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.218423+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:35:09.604 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.218673+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.219004+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.219359+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.219663+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.220022+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.220438+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.220714+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.220974+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.221247+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.221552+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.221744+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: audit 2024-09-10T18:35:09.222254+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:35:09.605 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:09 smithi151 bash[22565]: cluster 2024-09-10T18:35:09.230795+0000 mon.a (mon.0) 732 : cluster [INF] Manager daemon y is now available 2024-09-10T18:35:09.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:09.590+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f280c057be0 con 0x7f28141006a0 2024-09-10T18:35:09.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:09 smithi100 bash[18863]: [10/Sep/2024:18:35:09] ENGINE Bus STARTED 2024-09-10T18:35:10.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.198+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 19) v1 ==== 90554+0+0 (secure 0 0 0) 0x7f280c009590 con 0x7f28141006a0 2024-09-10T18:35:10.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.198+0000 7f2809ffb640 1 --2- 172.21.15.151:0/1958812932 >> v2:172.21.15.100:6800/2155730476 conn(0x7f28000756e0 0x7f2800077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:10.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.198+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 --> v2:172.21.15.100:6800/2155730476 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f2800009a60 con 0x7f28000756e0 2024-09-10T18:35:10.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.198+0000 7f2819431640 1 --2- 172.21.15.151:0/1958812932 >> v2:172.21.15.100:6800/2155730476 conn(0x7f28000756e0 0x7f2800077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:10.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.202+0000 7f2819431640 1 --2- 172.21.15.151:0/1958812932 >> v2:172.21.15.100:6800/2155730476 conn(0x7f28000756e0 0x7f2800077ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28100094b0 tx=0x7f2810009da0 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:10.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.210+0000 7f2809ffb640 1 -- 172.21.15.151:0/1958812932 <== mgr.24512 v2:172.21.15.100:6800/2155730476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f28100091f0 con 0x7f28000756e0 2024-09-10T18:35:10.229 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled alertmanager update... 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 >> v2:172.21.15.100:6800/2155730476 conn(0x7f28000756e0 msgr2=0x7f2800077ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 --2- 172.21.15.151:0/1958812932 >> v2:172.21.15.100:6800/2155730476 conn(0x7f28000756e0 0x7f2800077ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28100094b0 tx=0x7f2810009da0 comp rx=0 tx=0).stop 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28141006a0 msgr2=0x7f28141164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28141006a0 0x7f28141164a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f280c00ece0 tx=0x7f280c00b040 comp rx=0 tx=0).stop 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 shutdown_connections 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 --2- 172.21.15.151:0/1958812932 >> v2:172.21.15.100:6800/2155730476 conn(0x7f28000756e0 0x7f2800077ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28141006a0 0x7f28141164a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28140ffcf0 0x7f281410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:10.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 --2- 172.21.15.151:0/1958812932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28140fea20 0x7f281410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:10.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 >> 172.21.15.151:0/1958812932 conn(0x7f28140fa200 msgr2=0x7f28140fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:10.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 shutdown_connections 2024-09-10T18:35:10.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:10.214+0000 7f281b695640 1 -- 172.21.15.151:0/1958812932 wait complete. 2024-09-10T18:35:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.260802+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.293110+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.301221+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:35:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.302471+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:35:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.304875+0000 mon.b (mon.1) 32 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:35:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.306435+0000 mon.b (mon.1) 33 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: cluster 2024-09-10T18:35:09.310297+0000 mon.a (mon.0) 734 : cluster [DBG] Standby manager daemon x restarted 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: cluster 2024-09-10T18:35:09.310497+0000 mon.a (mon.0) 735 : cluster [DBG] Standby manager daemon x started 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.318624+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.336732+0000 mon.c (mon.2) 35 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.337287+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.364532+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:09.365062+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:10.054395+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:10.063821+0000 mon.a (mon.0) 739 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: cluster 2024-09-10T18:35:10.218254+0000 mon.a (mon.0) 740 : cluster [DBG] mgrmap e19: y(active, since 1.02466s), standbys: x 2024-09-10T18:35:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[18597]: audit 2024-09-10T18:35:10.229574+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.260802+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.293110+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.301221+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.302471+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.304875+0000 mon.b (mon.1) 32 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.306435+0000 mon.b (mon.1) 33 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: cluster 2024-09-10T18:35:09.310297+0000 mon.a (mon.0) 734 : cluster [DBG] Standby manager daemon x restarted 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: cluster 2024-09-10T18:35:09.310497+0000 mon.a (mon.0) 735 : cluster [DBG] Standby manager daemon x started 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.318624+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.336732+0000 mon.c (mon.2) 35 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.337287+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.364532+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:09.365062+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:35:10.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:10.054395+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:10.063821+0000 mon.a (mon.0) 739 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: cluster 2024-09-10T18:35:10.218254+0000 mon.a (mon.0) 740 : cluster [DBG] mgrmap e19: y(active, since 1.02466s), standbys: x 2024-09-10T18:35:10.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:10 smithi100 bash[31607]: audit 2024-09-10T18:35:10.229574+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.260802+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.293110+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.301221+0000 mon.b (mon.1) 30 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.302471+0000 mon.b (mon.1) 31 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.304875+0000 mon.b (mon.1) 32 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.306435+0000 mon.b (mon.1) 33 : audit [DBG] from='mgr.? 172.21.15.151:0/2326917778' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: cluster 2024-09-10T18:35:09.310297+0000 mon.a (mon.0) 734 : cluster [DBG] Standby manager daemon x restarted 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: cluster 2024-09-10T18:35:09.310497+0000 mon.a (mon.0) 735 : cluster [DBG] Standby manager daemon x started 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.318624+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.336732+0000 mon.c (mon.2) 35 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:35:10.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.337287+0000 mon.a (mon.0) 736 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:35:10.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.364532+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:35:10.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:09.365062+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:35:10.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:10.054395+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:10.063821+0000 mon.a (mon.0) 739 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:10.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: cluster 2024-09-10T18:35:10.218254+0000 mon.a (mon.0) 740 : cluster [DBG] mgrmap e19: y(active, since 1.02466s), standbys: x 2024-09-10T18:35:10.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:10 smithi151 bash[22565]: audit 2024-09-10T18:35:10.229574+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:11.133 DEBUG:teuthology.orchestra.run.smithi100:alertmanager.a> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T18:35:11.136 INFO:tasks.cephadm:Adding grafana.a on smithi151 2024-09-10T18:35:11.136 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi151=a' 2024-09-10T18:35:11.205 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:35:11.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:11 smithi151 bash[22565]: cephadm 2024-09-10T18:35:10.225798+0000 mgr.y (mgr.24512) 2 : cephadm [INF] Saving service alertmanager spec with placement smithi100=a;count:1 2024-09-10T18:35:11.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:11 smithi151 bash[22565]: cluster 2024-09-10T18:35:10.227193+0000 mgr.y (mgr.24512) 3 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:11.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[31607]: cephadm 2024-09-10T18:35:10.225798+0000 mgr.y (mgr.24512) 2 : cephadm [INF] Saving service alertmanager spec with placement smithi100=a;count:1 2024-09-10T18:35:11.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[31607]: cluster 2024-09-10T18:35:10.227193+0000 mgr.y (mgr.24512) 3 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:11.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18597]: cephadm 2024-09-10T18:35:10.225798+0000 mgr.y (mgr.24512) 2 : cephadm [INF] Saving service alertmanager spec with placement smithi100=a;count:1 2024-09-10T18:35:11.429 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18597]: cluster 2024-09-10T18:35:10.227193+0000 mgr.y (mgr.24512) 3 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:11.430 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: debug 2024-09-10T18:35:11.426+0000 7f182db0d640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:35:11] ENGINE Error in HTTPServer.serve 2024-09-10T18:35:11.430 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: Traceback (most recent call last): 2024-09-10T18:35:11.430 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:35:11.430 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: self._connections.run(self.expiration_interval) 2024-09-10T18:35:11.430 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:35:11.430 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: self._run(expiration_interval) 2024-09-10T18:35:11.430 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: s = self.context.wrap_socket( 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: return self.sslsocket_class._create( 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:35:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: self.do_handshake() 2024-09-10T18:35:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:35:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: self._sslobj.do_handshake() 2024-09-10T18:35:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:11 smithi100 bash[18863]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:35:12.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:35:11 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:35:12.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: cluster 2024-09-10T18:35:11.220409+0000 mgr.y (mgr.24512) 4 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:12.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: cluster 2024-09-10T18:35:11.269997+0000 mon.a (mon.0) 742 : cluster [DBG] mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:35:12.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: cephadm 2024-09-10T18:35:11.298717+0000 mgr.y (mgr.24512) 5 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Bus STARTING 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: cephadm 2024-09-10T18:35:11.425484+0000 mgr.y (mgr.24512) 6 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Serving on https://172.21.15.100:7150 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: cephadm 2024-09-10T18:35:11.425742+0000 mgr.y (mgr.24512) 7 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Bus STARTED 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: cephadm 2024-09-10T18:35:11.429694+0000 mgr.y (mgr.24512) 8 : cephadm [ERR] [10/Sep/2024:18:35:11] ENGINE Error in HTTPServer.serve 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: Traceback (most recent call last): 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: self._connections.run(self.expiration_interval) 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: self._run(expiration_interval) 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:35:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: s = self.context.wrap_socket( 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: return self.sslsocket_class._create( 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: self.do_handshake() 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: self._sslobj.do_handshake() 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: audit 2024-09-10T18:35:11.436360+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[18597]: audit 2024-09-10T18:35:11.860748+0000 mgr.y (mgr.24512) 9 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: cluster 2024-09-10T18:35:11.220409+0000 mgr.y (mgr.24512) 4 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:12.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: cluster 2024-09-10T18:35:11.269997+0000 mon.a (mon.0) 742 : cluster [DBG] mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: cephadm 2024-09-10T18:35:11.298717+0000 mgr.y (mgr.24512) 5 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Bus STARTING 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: cephadm 2024-09-10T18:35:11.425484+0000 mgr.y (mgr.24512) 6 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Serving on https://172.21.15.100:7150 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: cephadm 2024-09-10T18:35:11.425742+0000 mgr.y (mgr.24512) 7 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Bus STARTED 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: cephadm 2024-09-10T18:35:11.429694+0000 mgr.y (mgr.24512) 8 : cephadm [ERR] [10/Sep/2024:18:35:11] ENGINE Error in HTTPServer.serve 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: Traceback (most recent call last): 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: self._connections.run(self.expiration_interval) 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: self._run(expiration_interval) 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:35:12.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: s = self.context.wrap_socket( 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: return self.sslsocket_class._create( 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: self.do_handshake() 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: self._sslobj.do_handshake() 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: audit 2024-09-10T18:35:11.436360+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:12.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:12 smithi100 bash[31607]: audit 2024-09-10T18:35:11.860748+0000 mgr.y (mgr.24512) 9 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: cluster 2024-09-10T18:35:11.220409+0000 mgr.y (mgr.24512) 4 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: cluster 2024-09-10T18:35:11.269997+0000 mon.a (mon.0) 742 : cluster [DBG] mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: cephadm 2024-09-10T18:35:11.298717+0000 mgr.y (mgr.24512) 5 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Bus STARTING 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: cephadm 2024-09-10T18:35:11.425484+0000 mgr.y (mgr.24512) 6 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Serving on https://172.21.15.100:7150 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: cephadm 2024-09-10T18:35:11.425742+0000 mgr.y (mgr.24512) 7 : cephadm [INF] [10/Sep/2024:18:35:11] ENGINE Bus STARTED 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: cephadm 2024-09-10T18:35:11.429694+0000 mgr.y (mgr.24512) 8 : cephadm [ERR] [10/Sep/2024:18:35:11] ENGINE Error in HTTPServer.serve 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: Traceback (most recent call last): 2024-09-10T18:35:12.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: self._connections.run(self.expiration_interval) 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: self._run(expiration_interval) 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: s = self.context.wrap_socket( 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: return self.sslsocket_class._create( 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: self.do_handshake() 2024-09-10T18:35:12.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:35:12.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: self._sslobj.do_handshake() 2024-09-10T18:35:12.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:35:12.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: audit 2024-09-10T18:35:11.436360+0000 mon.a (mon.0) 743 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:12.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:12 smithi151 bash[22565]: audit 2024-09-10T18:35:11.860748+0000 mgr.y (mgr.24512) 9 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:14.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:14 smithi100 bash[31607]: cluster 2024-09-10T18:35:13.221148+0000 mgr.y (mgr.24512) 10 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:14.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:14 smithi100 bash[31607]: cluster 2024-09-10T18:35:13.286394+0000 mon.a (mon.0) 744 : cluster [DBG] mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:35:14.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:14 smithi100 bash[18597]: cluster 2024-09-10T18:35:13.221148+0000 mgr.y (mgr.24512) 10 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:14.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:14 smithi100 bash[18597]: cluster 2024-09-10T18:35:13.286394+0000 mon.a (mon.0) 744 : cluster [DBG] mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:35:14.551 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:14 smithi151 bash[22565]: cluster 2024-09-10T18:35:13.221148+0000 mgr.y (mgr.24512) 10 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:14.552 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:14 smithi151 bash[22565]: cluster 2024-09-10T18:35:13.286394+0000 mon.a (mon.0) 744 : cluster [DBG] mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T18:35:14.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:35:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:35:15.857 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:35:16.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.044247+0000 mon.a (mon.0) 745 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.057912+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: cluster 2024-09-10T18:35:15.221977+0000 mgr.y (mgr.24512) 11 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:16.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.676408+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.682421+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.687709+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.690606+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.693537+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.694378+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.696801+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.697470+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.699094+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.699716+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.701127+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.701637+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: cephadm 2024-09-10T18:35:15.702831+0000 mgr.y (mgr.24512) 12 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 3805M 2024-09-10T18:35:16.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:16 smithi151 bash[22565]: audit 2024-09-10T18:35:15.710518+0000 mon.a (mon.0) 755 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.044247+0000 mon.a (mon.0) 745 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.057912+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: cluster 2024-09-10T18:35:15.221977+0000 mgr.y (mgr.24512) 11 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:16.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.676408+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.682421+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.687709+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.690606+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.693537+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.694378+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.696801+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.697470+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.699094+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.699716+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.701127+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.701637+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: cephadm 2024-09-10T18:35:15.702831+0000 mgr.y (mgr.24512) 12 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 3805M 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[31607]: audit 2024-09-10T18:35:15.710518+0000 mon.a (mon.0) 755 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.044247+0000 mon.a (mon.0) 745 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.057912+0000 mon.a (mon.0) 746 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: cluster 2024-09-10T18:35:15.221977+0000 mgr.y (mgr.24512) 11 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.676408+0000 mon.a (mon.0) 747 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.682421+0000 mon.a (mon.0) 748 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.687709+0000 mon.a (mon.0) 749 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.690606+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.693537+0000 mon.a (mon.0) 750 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.694378+0000 mon.a (mon.0) 751 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.696801+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.697470+0000 mon.a (mon.0) 752 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.699094+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.699716+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.701127+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.701637+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:16.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: cephadm 2024-09-10T18:35:15.702831+0000 mgr.y (mgr.24512) 12 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 3805M 2024-09-10T18:35:16.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:16 smithi100 bash[18597]: audit 2024-09-10T18:35:15.710518+0000 mon.a (mon.0) 755 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.301919+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.311313+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.316012+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.316654+0000 mon.a (mon.0) 758 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.318286+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.318728+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.320054+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.320520+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.322046+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.322531+0000 mon.a (mon.0) 761 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.323794+0000 mgr.y (mgr.24512) 13 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 3805M 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.333737+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.335865+0000 mon.c (mon.2) 45 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.337403+0000 mon.c (mon.2) 46 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.339646+0000 mgr.y (mgr.24512) 14 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.339964+0000 mgr.y (mgr.24512) 15 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:35:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.476187+0000 mgr.y (mgr.24512) 16 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.480774+0000 mgr.y (mgr.24512) 17 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.607954+0000 mgr.y (mgr.24512) 18 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.617032+0000 mgr.y (mgr.24512) 19 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.301919+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.311313+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.316012+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.316654+0000 mon.a (mon.0) 758 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.318286+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.318728+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.320054+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.320520+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.322046+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.322531+0000 mon.a (mon.0) 761 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.323794+0000 mgr.y (mgr.24512) 13 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 3805M 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.333737+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.335865+0000 mon.c (mon.2) 45 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.337403+0000 mon.c (mon.2) 46 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.339646+0000 mgr.y (mgr.24512) 14 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.339964+0000 mgr.y (mgr.24512) 15 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.476187+0000 mgr.y (mgr.24512) 16 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.480774+0000 mgr.y (mgr.24512) 17 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.607954+0000 mgr.y (mgr.24512) 18 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:17.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.617032+0000 mgr.y (mgr.24512) 19 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.728269+0000 mgr.y (mgr.24512) 20 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.750362+0000 mgr.y (mgr.24512) 21 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.866928+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.877864+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.887422+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.898575+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: audit 2024-09-10T18:35:16.906908+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[18597]: cephadm 2024-09-10T18:35:16.913111+0000 mgr.y (mgr.24512) 22 : cephadm [INF] Deploying daemon node-exporter.a on smithi100 2024-09-10T18:35:17.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.728269+0000 mgr.y (mgr.24512) 20 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:17.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.750362+0000 mgr.y (mgr.24512) 21 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:17.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.866928+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.877864+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.887422+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.898575+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: audit 2024-09-10T18:35:16.906908+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[31607]: cephadm 2024-09-10T18:35:16.913111+0000 mgr.y (mgr.24512) 22 : cephadm [INF] Deploying daemon node-exporter.a on smithi100 2024-09-10T18:35:17.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.301919+0000 mon.a (mon.0) 756 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.311313+0000 mon.a (mon.0) 757 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.316012+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.316654+0000 mon.a (mon.0) 758 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.318286+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.318728+0000 mon.a (mon.0) 759 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.320054+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.320520+0000 mon.a (mon.0) 760 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.322046+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.322531+0000 mon.a (mon.0) 761 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.323794+0000 mgr.y (mgr.24512) 13 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 3805M 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.333737+0000 mon.a (mon.0) 762 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.335865+0000 mon.c (mon.2) 45 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.337403+0000 mon.c (mon.2) 46 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.339646+0000 mgr.y (mgr.24512) 14 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.conf 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.339964+0000 mgr.y (mgr.24512) 15 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.conf 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.476187+0000 mgr.y (mgr.24512) 16 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.480774+0000 mgr.y (mgr.24512) 17 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.607954+0000 mgr.y (mgr.24512) 18 : cephadm [INF] Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.617032+0000 mgr.y (mgr.24512) 19 : cephadm [INF] Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:17.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.728269+0000 mgr.y (mgr.24512) 20 : cephadm [INF] Updating smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.750362+0000 mgr.y (mgr.24512) 21 : cephadm [INF] Updating smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.866928+0000 mon.a (mon.0) 763 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.877864+0000 mon.a (mon.0) 764 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.887422+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.898575+0000 mon.a (mon.0) 766 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: audit 2024-09-10T18:35:16.906908+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:17 smithi151 bash[22565]: cephadm 2024-09-10T18:35:16.913111+0000 mgr.y (mgr.24512) 22 : cephadm [INF] Deploying daemon node-exporter.a on smithi100 2024-09-10T18:35:18.043 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:17 smithi100 systemd[1]: Started Ceph node-exporter.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:35:18.043 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:17 smithi100 bash[65654]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-09-10T18:35:18.472 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: v1.5.0: Pulling from prometheus/node-exporter 2024-09-10T18:35:18.675 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:18 smithi151 systemd[1]: Started Ceph node-exporter.b for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:35:18.676 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[56261]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-09-10T18:35:18.761 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: 22b70bddd3ac: Pulling fs layer 2024-09-10T18:35:18.761 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: 5c12815fee55: Pulling fs layer 2024-09-10T18:35:18.761 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: c0e87333d380: Pulling fs layer 2024-09-10T18:35:18.761 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: 22b70bddd3ac: Download complete 2024-09-10T18:35:18.761 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: 5c12815fee55: Download complete 2024-09-10T18:35:18.761 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: cluster 2024-09-10T18:35:17.223372+0000 mgr.y (mgr.24512) 23 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:35:18.761 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:17.675613+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.761 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:17.683674+0000 mon.a (mon.0) 769 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:17.691377+0000 mon.a (mon.0) 770 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: cephadm 2024-09-10T18:35:17.692542+0000 mgr.y (mgr.24512) 24 : cephadm [INF] Deploying daemon node-exporter.b on smithi151 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:18.483885+0000 mon.a (mon.0) 771 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:18.491975+0000 mon.a (mon.0) 772 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:18.501083+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:18.508904+0000 mon.a (mon.0) 774 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[18597]: audit 2024-09-10T18:35:18.520524+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: cluster 2024-09-10T18:35:17.223372+0000 mgr.y (mgr.24512) 23 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:17.675613+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:17.683674+0000 mon.a (mon.0) 769 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.762 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:17.691377+0000 mon.a (mon.0) 770 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: cephadm 2024-09-10T18:35:17.692542+0000 mgr.y (mgr.24512) 24 : cephadm [INF] Deploying daemon node-exporter.b on smithi151 2024-09-10T18:35:18.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:18.483885+0000 mon.a (mon.0) 771 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:18.491975+0000 mon.a (mon.0) 772 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:18.501083+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:18.508904+0000 mon.a (mon.0) 774 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:18.763 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[31607]: audit 2024-09-10T18:35:18.520524+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.042 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: c0e87333d380: Verifying Checksum 2024-09-10T18:35:19.043 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: c0e87333d380: Download complete 2024-09-10T18:35:19.043 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:18 smithi100 bash[65654]: 22b70bddd3ac: Pull complete 2024-09-10T18:35:19.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: cluster 2024-09-10T18:35:17.223372+0000 mgr.y (mgr.24512) 23 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:17.675613+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:17.683674+0000 mon.a (mon.0) 769 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:17.691377+0000 mon.a (mon.0) 770 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: cephadm 2024-09-10T18:35:17.692542+0000 mgr.y (mgr.24512) 24 : cephadm [INF] Deploying daemon node-exporter.b on smithi151 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:18.483885+0000 mon.a (mon.0) 771 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:18.491975+0000 mon.a (mon.0) 772 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:18.501083+0000 mon.a (mon.0) 773 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:18.508904+0000 mon.a (mon.0) 774 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:18 smithi151 bash[22565]: audit 2024-09-10T18:35:18.520524+0000 mon.a (mon.0) 775 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.748 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[22565]: cephadm 2024-09-10T18:35:18.536845+0000 mgr.y (mgr.24512) 25 : cephadm [INF] Deploying daemon alertmanager.a on smithi100 2024-09-10T18:35:19.748 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[22565]: audit 2024-09-10T18:35:19.296661+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.748 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: v1.5.0: Pulling from prometheus/node-exporter 2024-09-10T18:35:19.792 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:19 smithi100 bash[65654]: 5c12815fee55: Pull complete 2024-09-10T18:35:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:19 smithi100 bash[18597]: cephadm 2024-09-10T18:35:18.536845+0000 mgr.y (mgr.24512) 25 : cephadm [INF] Deploying daemon alertmanager.a on smithi100 2024-09-10T18:35:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:19 smithi100 bash[18597]: audit 2024-09-10T18:35:19.296661+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:19 smithi100 bash[31607]: cephadm 2024-09-10T18:35:18.536845+0000 mgr.y (mgr.24512) 25 : cephadm [INF] Deploying daemon alertmanager.a on smithi100 2024-09-10T18:35:19.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:19 smithi100 bash[31607]: audit 2024-09-10T18:35:19.296661+0000 mon.a (mon.0) 776 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- 172.21.15.151:0/405249069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 msgr2=0x7f554c066960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 --2- 172.21.15.151:0/405249069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 0x7f554c066960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f553c004950 tx=0x7f553c009da0 comp rx=0 tx=0).stop 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- 172.21.15.151:0/405249069 shutdown_connections 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 --2- 172.21.15.151:0/405249069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554c109030 0x7f554c10b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 --2- 172.21.15.151:0/405249069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 0x7f554c066960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 --2- 172.21.15.151:0/405249069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554c0fd670 0x7f554c065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- 172.21.15.151:0/405249069 >> 172.21.15.151:0/405249069 conn(0x7f554c0724b0 msgr2=0x7f554c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- 172.21.15.151:0/405249069 shutdown_connections 2024-09-10T18:35:19.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- 172.21.15.151:0/405249069 wait complete. 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 Processor -- start 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- start start 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 0x7f554c114fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554c0fd670 0x7f554c1154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554c109030 0x7f554c10f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c10f7a0 con 0x7f554c109030 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c10f910 con 0x7f554c066510 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f5551e57640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c10fad0 con 0x7f554c0fd670 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f554b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 0x7f554c114fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f554affd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554c0fd670 0x7f554c1154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f554b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 0x7f554c114fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47194/0 (socket says 172.21.15.151:47194) 2024-09-10T18:35:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f554bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554c109030 0x7f554c10f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:19.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.926+0000 7f554b7fe640 1 -- 172.21.15.151:0/1658947755 learned_addr learned my addr 172.21.15.151:0/1658947755 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:35:19.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f554affd640 1 -- 172.21.15.151:0/1658947755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 msgr2=0x7f554c114fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:19.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f554affd640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 0x7f554c114fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:19.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f554affd640 1 -- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554c109030 msgr2=0x7f554c10f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:19.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f554affd640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554c109030 0x7f554c10f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:19.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f554affd640 1 -- 172.21.15.151:0/1658947755 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c005040 con 0x7f554c0fd670 2024-09-10T18:35:19.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f554bfff640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554c109030 0x7f554c10f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:35:19.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f554affd640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554c0fd670 0x7f554c1154f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f553c02a4a0 tx=0x7f553c009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:19.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f5533fff640 1 -- 172.21.15.151:0/1658947755 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c0098d0 con 0x7f554c0fd670 2024-09-10T18:35:19.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c10fd50 con 0x7f554c0fd670 2024-09-10T18:35:19.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f5533fff640 1 -- 172.21.15.151:0/1658947755 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f553c0325e0 con 0x7f554c0fd670 2024-09-10T18:35:19.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f5533fff640 1 -- 172.21.15.151:0/1658947755 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c033690 con 0x7f554c0fd670 2024-09-10T18:35:19.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c11bfa0 con 0x7f554c0fd670 2024-09-10T18:35:19.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.930+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554c0fbd60 con 0x7f554c0fd670 2024-09-10T18:35:19.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.934+0000 7f5533fff640 1 -- 172.21.15.151:0/1658947755 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f553c030020 con 0x7f554c0fd670 2024-09-10T18:35:19.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.934+0000 7f5533fff640 1 --2- 172.21.15.151:0/1658947755 >> v2:172.21.15.100:6800/2155730476 conn(0x7f553406eec0 0x7f5534071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:19.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.934+0000 7f554b7fe640 1 --2- 172.21.15.151:0/1658947755 >> v2:172.21.15.100:6800/2155730476 conn(0x7f553406eec0 0x7f5534071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:19.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.934+0000 7f5533fff640 1 -- 172.21.15.151:0/1658947755 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f553c0adb90 con 0x7f554c0fd670 2024-09-10T18:35:19.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.934+0000 7f554b7fe640 1 --2- 172.21.15.151:0/1658947755 >> v2:172.21.15.100:6800/2155730476 conn(0x7f553406eec0 0x7f5534071370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55400041d0 tx=0x7f554000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:19.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:19.942+0000 7f5533fff640 1 -- 172.21.15.151:0/1658947755 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f553c07ab20 con 0x7f554c0fd670 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: 22b70bddd3ac: Pulling fs layer 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: 5c12815fee55: Pulling fs layer 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: c0e87333d380: Pulling fs layer 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: 22b70bddd3ac: Verifying Checksum 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: 22b70bddd3ac: Download complete 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: 5c12815fee55: Verifying Checksum 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:19 smithi151 bash[56261]: 5c12815fee55: Download complete 2024-09-10T18:35:20.129 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:20 smithi151 bash[56261]: c0e87333d380: Verifying Checksum 2024-09-10T18:35:20.130 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:20 smithi151 bash[56261]: c0e87333d380: Download complete 2024-09-10T18:35:20.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.142+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 --> v2:172.21.15.100:6800/2155730476 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f554c1106b0 con 0x7f553406eec0 2024-09-10T18:35:20.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5533fff640 1 -- 172.21.15.151:0/1658947755 <== mgr.24512 v2:172.21.15.100:6800/2155730476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f554000f040 con 0x7f553406eec0 2024-09-10T18:35:20.179 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled grafana update... 2024-09-10T18:35:20.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 >> v2:172.21.15.100:6800/2155730476 conn(0x7f553406eec0 msgr2=0x7f5534071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:20.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 --2- 172.21.15.151:0/1658947755 >> v2:172.21.15.100:6800/2155730476 conn(0x7f553406eec0 0x7f5534071370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55400041d0 tx=0x7f554000b040 comp rx=0 tx=0).stop 2024-09-10T18:35:20.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554c0fd670 msgr2=0x7f554c1154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:20.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554c0fd670 0x7f554c1154f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f553c02a4a0 tx=0x7f553c009da0 comp rx=0 tx=0).stop 2024-09-10T18:35:20.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 shutdown_connections 2024-09-10T18:35:20.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 --2- 172.21.15.151:0/1658947755 >> v2:172.21.15.100:6800/2155730476 conn(0x7f553406eec0 0x7f5534071370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:20.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554c109030 0x7f554c10f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:20.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554c0fd670 0x7f554c1154f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:20.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 --2- 172.21.15.151:0/1658947755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f554c066510 0x7f554c114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:20.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 >> 172.21.15.151:0/1658947755 conn(0x7f554c0724b0 msgr2=0x7f554c109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:20.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 shutdown_connections 2024-09-10T18:35:20.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:20.162+0000 7f5551e57640 1 -- 172.21.15.151:0/1658947755 wait complete. 2024-09-10T18:35:20.355 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:20 smithi100 bash[65654]: c0e87333d380: Pull complete 2024-09-10T18:35:20.425 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:20 smithi151 bash[56261]: 22b70bddd3ac: Pull complete 2024-09-10T18:35:20.690 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:20 smithi100 bash[65654]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-09-10T18:35:20.690 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:20 smithi100 bash[65654]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:35:20.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:20 smithi100 bash[18597]: cluster 2024-09-10T18:35:19.223988+0000 mgr.y (mgr.24512) 26 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:35:20.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:20 smithi100 bash[18597]: audit 2024-09-10T18:35:20.178664+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:20.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:20 smithi100 bash[31607]: cluster 2024-09-10T18:35:19.223988+0000 mgr.y (mgr.24512) 26 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:35:20.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:20 smithi100 bash[31607]: audit 2024-09-10T18:35:20.178664+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:20.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:20 smithi151 bash[22565]: cluster 2024-09-10T18:35:19.223988+0000 mgr.y (mgr.24512) 26 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:35:20.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:20 smithi151 bash[22565]: audit 2024-09-10T18:35:20.178664+0000 mon.a (mon.0) 777 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:21.321 DEBUG:teuthology.orchestra.run.smithi151:grafana.a> sudo journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T18:35:21.323 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T18:35:21.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:35:21.458 INFO:journalctl@ceph.grafana.a.smithi151.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-09-10T18:35:21.675 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:21 smithi151 bash[56261]: 5c12815fee55: Pull complete 2024-09-10T18:35:21.980 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:35:21 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:35:21.980 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:21 smithi151 bash[56261]: c0e87333d380: Pull complete 2024-09-10T18:35:21.980 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:21 smithi151 bash[56261]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-09-10T18:35:21.981 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:21 smithi151 bash[22565]: audit 2024-09-10T18:35:20.164917+0000 mgr.y (mgr.24512) 27 : audit [DBG] from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:21.981 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:21 smithi151 bash[22565]: cephadm 2024-09-10T18:35:20.168310+0000 mgr.y (mgr.24512) 28 : cephadm [INF] Saving service grafana spec with placement smithi151=a;count:1 2024-09-10T18:35:22.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:21 smithi100 bash[18597]: audit 2024-09-10T18:35:20.164917+0000 mgr.y (mgr.24512) 27 : audit [DBG] from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:22.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:21 smithi100 bash[18597]: cephadm 2024-09-10T18:35:20.168310+0000 mgr.y (mgr.24512) 28 : cephadm [INF] Saving service grafana spec with placement smithi151=a;count:1 2024-09-10T18:35:22.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:21 smithi100 bash[31607]: audit 2024-09-10T18:35:20.164917+0000 mgr.y (mgr.24512) 27 : audit [DBG] from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:22.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:21 smithi100 bash[31607]: cephadm 2024-09-10T18:35:20.168310+0000 mgr.y (mgr.24512) 28 : cephadm [INF] Saving service grafana spec with placement smithi151=a;count:1 2024-09-10T18:35:22.425 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:21 smithi151 bash[56261]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:35:23.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:22 smithi100 bash[18597]: cluster 2024-09-10T18:35:21.225565+0000 mgr.y (mgr.24512) 29 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:35:23.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:22 smithi100 bash[18597]: audit 2024-09-10T18:35:21.871352+0000 mgr.y (mgr.24512) 30 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:23.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:22 smithi100 bash[31607]: cluster 2024-09-10T18:35:21.225565+0000 mgr.y (mgr.24512) 29 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:35:23.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:22 smithi100 bash[31607]: audit 2024-09-10T18:35:21.871352+0000 mgr.y (mgr.24512) 30 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:23.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:22 smithi151 bash[22565]: cluster 2024-09-10T18:35:21.225565+0000 mgr.y (mgr.24512) 29 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:35:23.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:22 smithi151 bash[22565]: audit 2024-09-10T18:35:21.871352+0000 mgr.y (mgr.24512) 30 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:23.543 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.202Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.202Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.203Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.203Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.204Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.204Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.204Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.204Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:35:23.544 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:35:23.545 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:35:23.546 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:35:23.547 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:35:23.548 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:35:23.548 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.205Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:35:23.548 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.206Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:35:23.548 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:35:23 smithi100 bash[65654]: ts=2024-09-10T18:35:23.206Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.548Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.548Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.549Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.549Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:35:23.926 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:35:23.927 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:35:23.928 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:35:23.929 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.550Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:35:23.930 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:35:23 smithi151 bash[56261]: ts=2024-09-10T18:35:23.551Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:35:24.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:35:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:35:24.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:24 smithi151 bash[22565]: cluster 2024-09-10T18:35:23.226233+0000 mgr.y (mgr.24512) 31 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:35:24.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:24 smithi151 bash[22565]: audit 2024-09-10T18:35:24.313424+0000 mon.c (mon.2) 47 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:24 smithi100 bash[18597]: cluster 2024-09-10T18:35:23.226233+0000 mgr.y (mgr.24512) 31 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:35:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:24 smithi100 bash[18597]: audit 2024-09-10T18:35:24.313424+0000 mon.c (mon.2) 47 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:24 smithi100 bash[31607]: cluster 2024-09-10T18:35:23.226233+0000 mgr.y (mgr.24512) 31 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:35:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:24 smithi100 bash[31607]: audit 2024-09-10T18:35:24.313424+0000 mon.c (mon.2) 47 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:26 smithi100 bash[18597]: cluster 2024-09-10T18:35:25.227622+0000 mgr.y (mgr.24512) 32 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:35:27.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:26 smithi100 bash[31607]: cluster 2024-09-10T18:35:25.227622+0000 mgr.y (mgr.24512) 32 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:35:27.134 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:27.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:26 smithi151 bash[22565]: cluster 2024-09-10T18:35:25.227622+0000 mgr.y (mgr.24512) 32 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:35:29.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:28 smithi100 bash[18597]: cluster 2024-09-10T18:35:27.229001+0000 mgr.y (mgr.24512) 33 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:35:29.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:28 smithi100 bash[31607]: cluster 2024-09-10T18:35:27.229001+0000 mgr.y (mgr.24512) 33 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:35:29.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:28 smithi151 bash[22565]: cluster 2024-09-10T18:35:27.229001+0000 mgr.y (mgr.24512) 33 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:35:30.043 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:29 smithi100 systemd[1]: Started Ceph alertmanager.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:35:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[31607]: cluster 2024-09-10T18:35:29.229770+0000 mgr.y (mgr.24512) 34 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[31607]: audit 2024-09-10T18:35:30.011900+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[31607]: audit 2024-09-10T18:35:30.020360+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[31607]: audit 2024-09-10T18:35:30.026996+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[31607]: audit 2024-09-10T18:35:30.033629+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[31607]: audit 2024-09-10T18:35:30.076245+0000 mon.c (mon.2) 48 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[18597]: cluster 2024-09-10T18:35:29.229770+0000 mgr.y (mgr.24512) 34 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[18597]: audit 2024-09-10T18:35:30.011900+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[18597]: audit 2024-09-10T18:35:30.020360+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[18597]: audit 2024-09-10T18:35:30.026996+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[18597]: audit 2024-09-10T18:35:30.033629+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:30 smithi100 bash[18597]: audit 2024-09-10T18:35:30.076245+0000 mon.c (mon.2) 48 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:31.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:30 smithi151 bash[22565]: cluster 2024-09-10T18:35:29.229770+0000 mgr.y (mgr.24512) 34 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:31.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:30 smithi151 bash[22565]: audit 2024-09-10T18:35:30.011900+0000 mon.a (mon.0) 778 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:30 smithi151 bash[22565]: audit 2024-09-10T18:35:30.020360+0000 mon.a (mon.0) 779 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:30 smithi151 bash[22565]: audit 2024-09-10T18:35:30.026996+0000 mon.a (mon.0) 780 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:30 smithi151 bash[22565]: audit 2024-09-10T18:35:30.033629+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:31.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:30 smithi151 bash[22565]: audit 2024-09-10T18:35:30.076245+0000 mon.c (mon.2) 48 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:32.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- 172.21.15.100:0/1287457894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0ff4a0 msgr2=0x7eff1c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 --2- 172.21.15.100:0/1287457894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0ff4a0 0x7eff1c10b8c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009d00 tx=0x7eff0c004d10 comp rx=0 tx=0).stop 2024-09-10T18:35:32.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- 172.21.15.100:0/1287457894 shutdown_connections 2024-09-10T18:35:32.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 --2- 172.21.15.100:0/1287457894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0ff4a0 0x7eff1c10b8c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 --2- 172.21.15.100:0/1287457894 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c0feaf0 0x7eff1c0fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 --2- 172.21.15.100:0/1287457894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff1c104b30 0x7eff1c104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- 172.21.15.100:0/1287457894 >> 172.21.15.100:0/1287457894 conn(0x7eff1c0723b0 msgr2=0x7eff1c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:32.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- 172.21.15.100:0/1287457894 shutdown_connections 2024-09-10T18:35:32.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- 172.21.15.100:0/1287457894 wait complete. 2024-09-10T18:35:32.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 Processor -- start 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- start start 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0feaf0 0x7eff1c114d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff1c0ff4a0 0x7eff1c115240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c104b30 0x7eff1c10eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c10f4a0 con 0x7eff1c0feaf0 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c10f610 con 0x7eff1c0ff4a0 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff23579640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c10f7d0 con 0x7eff1c104b30 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c104b30 0x7eff1c10eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21315640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0feaf0 0x7eff1c114d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c104b30 0x7eff1c10eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:50740/0 (socket says 172.21.15.100:50740) 2024-09-10T18:35:32.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 -- 172.21.15.100:0/3987696612 learned_addr learned my addr 172.21.15.100:0/3987696612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff20b14640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff1c0ff4a0 0x7eff1c115240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 -- 172.21.15.100:0/3987696612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff1c0ff4a0 msgr2=0x7eff1c115240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff1c0ff4a0 0x7eff1c115240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 -- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0feaf0 msgr2=0x7eff1c114d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0feaf0 0x7eff1c114d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 -- 172.21.15.100:0/3987696612 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0098b0 con 0x7eff1c104b30 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21315640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0feaf0 0x7eff1c114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff21b16640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c104b30 0x7eff1c10eed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7eff0c004a30 tx=0x7eff0c004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff19ffb640 1 -- 172.21.15.100:0/3987696612 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c038070 con 0x7eff1c104b30 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.089+0000 7eff19ffb640 1 -- 172.21.15.100:0/3987696612 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7eff0c033e90 con 0x7eff1c104b30 2024-09-10T18:35:32.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.093+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c10fa50 con 0x7eff1c104b30 2024-09-10T18:35:32.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.093+0000 7eff19ffb640 1 -- 172.21.15.100:0/3987696612 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c03c990 con 0x7eff1c104b30 2024-09-10T18:35:32.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.093+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c11bd30 con 0x7eff1c104b30 2024-09-10T18:35:32.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.093+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c1001f0 con 0x7eff1c104b30 2024-09-10T18:35:32.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.093+0000 7eff19ffb640 1 -- 172.21.15.100:0/3987696612 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7eff0c030070 con 0x7eff1c104b30 2024-09-10T18:35:32.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.097+0000 7eff19ffb640 1 --2- 172.21.15.100:0/3987696612 >> v2:172.21.15.100:6800/2155730476 conn(0x7eff0406eda0 0x7eff04071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:32.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.097+0000 7eff19ffb640 1 -- 172.21.15.100:0/3987696612 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6168+0+0 (secure 0 0 0) 0x7eff0c0aedc0 con 0x7eff1c104b30 2024-09-10T18:35:32.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.097+0000 7eff21315640 1 --2- 172.21.15.100:0/3987696612 >> v2:172.21.15.100:6800/2155730476 conn(0x7eff0406eda0 0x7eff04071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:32.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.097+0000 7eff21315640 1 --2- 172.21.15.100:0/3987696612 >> v2:172.21.15.100:6800/2155730476 conn(0x7eff0406eda0 0x7eff04071250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff14009fd0 tx=0x7eff14009e50 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:32.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.105+0000 7eff19ffb640 1 -- 172.21.15.100:0/3987696612 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7eff0c07bca0 con 0x7eff1c104b30 2024-09-10T18:35:32.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:35:31 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.002Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.002Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.005Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.100 port=9094 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.006Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.050Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.051Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.055Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:35:32.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[66166]: ts=2024-09-10T18:35:32.055Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:35:32.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.345+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7eff1c04bf90 con 0x7eff1c104b30 2024-09-10T18:35:32.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.353+0000 7eff19ffb640 1 -- 172.21.15.100:0/3987696612 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (secure 0 0 0) 0x7eff0c07fe60 con 0x7eff1c104b30 2024-09-10T18:35:32.359 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-10T18:35:32.359 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB0keBmfqPyFBAAH779aJ8CzxOKh0WQnY+vDQ== 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 >> v2:172.21.15.100:6800/2155730476 conn(0x7eff0406eda0 msgr2=0x7eff04071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 --2- 172.21.15.100:0/3987696612 >> v2:172.21.15.100:6800/2155730476 conn(0x7eff0406eda0 0x7eff04071250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff14009fd0 tx=0x7eff14009e50 comp rx=0 tx=0).stop 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c104b30 msgr2=0x7eff1c10eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c104b30 0x7eff1c10eed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7eff0c004a30 tx=0x7eff0c004d10 comp rx=0 tx=0).stop 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 shutdown_connections 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 --2- 172.21.15.100:0/3987696612 >> v2:172.21.15.100:6800/2155730476 conn(0x7eff0406eda0 0x7eff04071250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff1c104b30 0x7eff1c10eed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff1c0ff4a0 0x7eff1c115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 --2- 172.21.15.100:0/3987696612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff1c0feaf0 0x7eff1c114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 >> 172.21.15.100:0/3987696612 conn(0x7eff1c0723b0 msgr2=0x7eff1c103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 shutdown_connections 2024-09-10T18:35:32.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:32.357+0000 7eff23579640 1 -- 172.21.15.100:0/3987696612 wait complete. 2024-09-10T18:35:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[18597]: cluster 2024-09-10T18:35:31.231350+0000 mgr.y (mgr.24512) 35 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[18597]: audit 2024-09-10T18:35:31.881555+0000 mgr.y (mgr.24512) 36 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[18597]: audit 2024-09-10T18:35:32.350712+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.100:0/3987696612' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[18597]: audit 2024-09-10T18:35:32.351203+0000 mon.a (mon.0) 782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[18597]: audit 2024-09-10T18:35:32.359798+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:35:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[31607]: cluster 2024-09-10T18:35:31.231350+0000 mgr.y (mgr.24512) 35 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[31607]: audit 2024-09-10T18:35:31.881555+0000 mgr.y (mgr.24512) 36 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[31607]: audit 2024-09-10T18:35:32.350712+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.100:0/3987696612' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[31607]: audit 2024-09-10T18:35:32.351203+0000 mon.a (mon.0) 782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:32 smithi100 bash[31607]: audit 2024-09-10T18:35:32.359798+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:35:33.111 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:35:33.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T18:35:33.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T18:35:33.138 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:32 smithi151 bash[22565]: cluster 2024-09-10T18:35:31.231350+0000 mgr.y (mgr.24512) 35 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:32 smithi151 bash[22565]: audit 2024-09-10T18:35:31.881555+0000 mgr.y (mgr.24512) 36 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:32 smithi151 bash[22565]: audit 2024-09-10T18:35:32.350712+0000 mon.c (mon.2) 49 : audit [INF] from='client.? 172.21.15.100:0/3987696612' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:32 smithi151 bash[22565]: audit 2024-09-10T18:35:32.351203+0000 mon.a (mon.0) 782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:33.152 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:32 smithi151 bash[22565]: audit 2024-09-10T18:35:32.359798+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:35:34.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:34 smithi100 bash[66166]: ts=2024-09-10T18:35:34.007Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000516177s 2024-09-10T18:35:34.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:34 smithi151 bash[22565]: cluster 2024-09-10T18:35:33.232190+0000 mgr.y (mgr.24512) 37 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:34.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:34 smithi151 bash[22565]: audit 2024-09-10T18:35:34.309002+0000 mon.a (mon.0) 784 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:34.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:35:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:35:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:34 smithi100 bash[18597]: cluster 2024-09-10T18:35:33.232190+0000 mgr.y (mgr.24512) 37 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:34 smithi100 bash[18597]: audit 2024-09-10T18:35:34.309002+0000 mon.a (mon.0) 784 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:35.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:34 smithi100 bash[31607]: cluster 2024-09-10T18:35:33.232190+0000 mgr.y (mgr.24512) 37 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:35.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:34 smithi100 bash[31607]: audit 2024-09-10T18:35:34.309002+0000 mon.a (mon.0) 784 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:36.904 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:35:37.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: cluster 2024-09-10T18:35:35.233741+0000 mgr.y (mgr.24512) 38 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: audit 2024-09-10T18:35:35.852541+0000 mon.a (mon.0) 785 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: audit 2024-09-10T18:35:35.864866+0000 mon.a (mon.0) 786 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: audit 2024-09-10T18:35:36.782692+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: audit 2024-09-10T18:35:36.796104+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: audit 2024-09-10T18:35:36.798245+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: audit 2024-09-10T18:35:36.799463+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:36 smithi151 bash[22565]: audit 2024-09-10T18:35:36.806410+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18863]: [10/Sep/2024:18:35:36] ENGINE Bus STOPPING 2024-09-10T18:35:37.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18863]: [10/Sep/2024:18:35:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:35:37.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18863]: [10/Sep/2024:18:35:36] ENGINE Bus STOPPED 2024-09-10T18:35:37.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18863]: [10/Sep/2024:18:35:36] ENGINE Bus STARTING 2024-09-10T18:35:37.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18863]: [10/Sep/2024:18:35:37] ENGINE Serving on http://:::9283 2024-09-10T18:35:37.293 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18863]: [10/Sep/2024:18:35:37] ENGINE Bus STARTED 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: cluster 2024-09-10T18:35:35.233741+0000 mgr.y (mgr.24512) 38 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: audit 2024-09-10T18:35:35.852541+0000 mon.a (mon.0) 785 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: audit 2024-09-10T18:35:35.864866+0000 mon.a (mon.0) 786 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: audit 2024-09-10T18:35:36.782692+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: audit 2024-09-10T18:35:36.796104+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: audit 2024-09-10T18:35:36.798245+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: audit 2024-09-10T18:35:36.799463+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[18597]: audit 2024-09-10T18:35:36.806410+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: cluster 2024-09-10T18:35:35.233741+0000 mgr.y (mgr.24512) 38 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: audit 2024-09-10T18:35:35.852541+0000 mon.a (mon.0) 785 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: audit 2024-09-10T18:35:35.864866+0000 mon.a (mon.0) 786 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: audit 2024-09-10T18:35:36.782692+0000 mon.a (mon.0) 787 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: audit 2024-09-10T18:35:36.796104+0000 mon.a (mon.0) 788 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: audit 2024-09-10T18:35:36.798245+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: audit 2024-09-10T18:35:36.799463+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:36 smithi100 bash[31607]: audit 2024-09-10T18:35:36.806410+0000 mon.a (mon.0) 789 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:37 smithi151 bash[22565]: audit 2024-09-10T18:35:36.869852+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:37 smithi151 bash[22565]: audit 2024-09-10T18:35:36.878404+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:37 smithi151 bash[22565]: audit 2024-09-10T18:35:36.882871+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:35:38.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:37 smithi151 bash[22565]: audit 2024-09-10T18:35:36.883720+0000 mgr.y (mgr.24512) 39 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:35:38.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:37 smithi151 bash[22565]: audit 2024-09-10T18:35:36.890434+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:37 smithi151 bash[22565]: cephadm 2024-09-10T18:35:36.917333+0000 mgr.y (mgr.24512) 40 : cephadm [INF] Deploying daemon grafana.a on smithi151 2024-09-10T18:35:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18597]: audit 2024-09-10T18:35:36.869852+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18597]: audit 2024-09-10T18:35:36.878404+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18597]: audit 2024-09-10T18:35:36.882871+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:35:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18597]: audit 2024-09-10T18:35:36.883720+0000 mgr.y (mgr.24512) 39 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:35:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18597]: audit 2024-09-10T18:35:36.890434+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[18597]: cephadm 2024-09-10T18:35:36.917333+0000 mgr.y (mgr.24512) 40 : cephadm [INF] Deploying daemon grafana.a on smithi151 2024-09-10T18:35:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[31607]: audit 2024-09-10T18:35:36.869852+0000 mon.a (mon.0) 790 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[31607]: audit 2024-09-10T18:35:36.878404+0000 mon.a (mon.0) 791 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[31607]: audit 2024-09-10T18:35:36.882871+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:35:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[31607]: audit 2024-09-10T18:35:36.883720+0000 mgr.y (mgr.24512) 39 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:35:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[31607]: audit 2024-09-10T18:35:36.890434+0000 mon.a (mon.0) 792 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:35:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:37 smithi100 bash[31607]: cephadm 2024-09-10T18:35:36.917333+0000 mgr.y (mgr.24512) 40 : cephadm [INF] Deploying daemon grafana.a on smithi151 2024-09-10T18:35:39.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:38 smithi151 bash[22565]: cluster 2024-09-10T18:35:37.234475+0000 mgr.y (mgr.24512) 41 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:38 smithi100 bash[18597]: cluster 2024-09-10T18:35:37.234475+0000 mgr.y (mgr.24512) 41 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:38 smithi100 bash[31607]: cluster 2024-09-10T18:35:37.234475+0000 mgr.y (mgr.24512) 41 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:40.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:39 smithi151 bash[22565]: audit 2024-09-10T18:35:39.314569+0000 mon.c (mon.2) 53 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:40.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:39 smithi100 bash[18597]: audit 2024-09-10T18:35:39.314569+0000 mon.c (mon.2) 53 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:40.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:39 smithi100 bash[31607]: audit 2024-09-10T18:35:39.314569+0000 mon.c (mon.2) 53 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:41.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:40 smithi151 bash[22565]: cluster 2024-09-10T18:35:39.235196+0000 mgr.y (mgr.24512) 42 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:41.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:40 smithi100 bash[18597]: cluster 2024-09-10T18:35:39.235196+0000 mgr.y (mgr.24512) 42 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:41.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:40 smithi100 bash[31607]: cluster 2024-09-10T18:35:39.235196+0000 mgr.y (mgr.24512) 42 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:41.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 -- 172.21.15.151:0/2799769706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c1017e0 msgr2=0x7f4d2c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:41.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 --2- 172.21.15.151:0/2799769706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c1017e0 0x7f4d2c101bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4d28004950 tx=0x7f4d28009da0 comp rx=0 tx=0).stop 2024-09-10T18:35:41.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 -- 172.21.15.151:0/2799769706 shutdown_connections 2024-09-10T18:35:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 --2- 172.21.15.151:0/2799769706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d2c0fc2e0 0x7f4d2c108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 --2- 172.21.15.151:0/2799769706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c0fb950 0x7f4d2c0fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 --2- 172.21.15.151:0/2799769706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c1017e0 0x7f4d2c101bb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 -- 172.21.15.151:0/2799769706 >> 172.21.15.151:0/2799769706 conn(0x7f4d2c0734f0 msgr2=0x7f4d2c0738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 -- 172.21.15.151:0/2799769706 shutdown_connections 2024-09-10T18:35:41.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.910+0000 7f4d33bea640 1 -- 172.21.15.151:0/2799769706 wait complete. 2024-09-10T18:35:41.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 Processor -- start 2024-09-10T18:35:41.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 -- start start 2024-09-10T18:35:41.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d2c0fb950 0x7f4d2c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:41.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c0fc2e0 0x7f4d2c10e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:41.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1017e0 0x7f4d2c10e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c1190f0 con 0x7f4d2c1017e0 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c119260 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c119420 con 0x7f4d2c0fb950 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c0fc2e0 0x7f4d2c10e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c0fc2e0 0x7f4d2c10e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36770/0 (socket says 172.21.15.151:36770) 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 -- 172.21.15.151:0/419938161 learned_addr learned my addr 172.21.15.151:0/419938161 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31986640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d2c0fb950 0x7f4d2c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d32187640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1017e0 0x7f4d2c10e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 -- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d2c0fb950 msgr2=0x7f4d2c10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d2c0fb950 0x7f4d2c10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 -- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1017e0 msgr2=0x7f4d2c10e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:41.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1017e0 0x7f4d2c10e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 -- 172.21.15.151:0/419938161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28005040 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d32187640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1017e0 0x7f4d2c10e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:35:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31986640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d2c0fb950 0x7f4d2c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:35:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d31185640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c0fc2e0 0x7f4d2c10e3b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c002b00 tx=0x7f4d1c004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d227fc640 1 -- 172.21.15.151:0/419938161 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c007c30 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c119700 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d227fc640 1 -- 172.21.15.151:0/419938161 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f4d1c00f040 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d227fc640 1 -- 172.21.15.151:0/419938161 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c009320 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.914+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c119c20 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.918+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c0fcf00 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.918+0000 7f4d227fc640 1 -- 172.21.15.151:0/419938161 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f4d1c009750 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.918+0000 7f4d227fc640 1 --2- 172.21.15.151:0/419938161 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4d1806eed0 0x7f4d18071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.918+0000 7f4d227fc640 1 -- 172.21.15.151:0/419938161 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f4d1c08e3a0 con 0x7f4d2c0fc2e0 2024-09-10T18:35:41.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.918+0000 7f4d31986640 1 --2- 172.21.15.151:0/419938161 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4d1806eed0 0x7f4d18071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:41.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.918+0000 7f4d31986640 1 --2- 172.21.15.151:0/419938161 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4d1806eed0 0x7f4d18071380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d2802d040 tx=0x7f4d28035040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:41.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:41.926+0000 7f4d227fc640 1 -- 172.21.15.151:0/419938161 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4d1c05b2f0 con 0x7f4d2c0fc2e0 2024-09-10T18:35:42.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:35:41 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:35:42.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.202+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4d2c065880 con 0x7f4d2c0fc2e0 2024-09-10T18:35:42.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d227fc640 1 -- 172.21.15.151:0/419938161 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v19) v1 ==== 170+0+59 (secure 0 0 0) 0x7f4d1c05ac90 con 0x7f4d2c0fc2e0 2024-09-10T18:35:42.231 INFO:teuthology.orchestra.run.smithi151.stdout:[client.1] 2024-09-10T18:35:42.231 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQB+keBmzr46DRAAHdIJggU0KpMjmBv+jQpefQ== 2024-09-10T18:35:42.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4d1806eed0 msgr2=0x7f4d18071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:42.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 --2- 172.21.15.151:0/419938161 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4d1806eed0 0x7f4d18071380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d2802d040 tx=0x7f4d28035040 comp rx=0 tx=0).stop 2024-09-10T18:35:42.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c0fc2e0 msgr2=0x7f4d2c10e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:42.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c0fc2e0 0x7f4d2c10e3b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c002b00 tx=0x7f4d1c004600 comp rx=0 tx=0).stop 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 shutdown_connections 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 --2- 172.21.15.151:0/419938161 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4d1806eed0 0x7f4d18071380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d2c1017e0 0x7f4d2c10e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d2c0fc2e0 0x7f4d2c10e3b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 --2- 172.21.15.151:0/419938161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d2c0fb950 0x7f4d2c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.214+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 >> 172.21.15.151:0/419938161 conn(0x7f4d2c0734f0 msgr2=0x7f4d2c0fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.218+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 shutdown_connections 2024-09-10T18:35:42.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-10T18:35:42.218+0000 7f4d33bea640 1 -- 172.21.15.151:0/419938161 wait complete. 2024-09-10T18:35:42.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[66166]: ts=2024-09-10T18:35:42.010Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003584973s 2024-09-10T18:35:43.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:42 smithi151 bash[22565]: cluster 2024-09-10T18:35:41.236708+0000 mgr.y (mgr.24512) 43 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:43.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:42 smithi151 bash[22565]: audit 2024-09-10T18:35:41.892625+0000 mgr.y (mgr.24512) 44 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:43.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:42 smithi151 bash[22565]: audit 2024-09-10T18:35:42.207097+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.151:0/419938161' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:43.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:42 smithi151 bash[22565]: audit 2024-09-10T18:35:42.221602+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:43.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:42 smithi151 bash[22565]: audit 2024-09-10T18:35:42.231468+0000 mon.a (mon.0) 794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:35:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[18597]: cluster 2024-09-10T18:35:41.236708+0000 mgr.y (mgr.24512) 43 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[18597]: audit 2024-09-10T18:35:41.892625+0000 mgr.y (mgr.24512) 44 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[18597]: audit 2024-09-10T18:35:42.207097+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.151:0/419938161' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[18597]: audit 2024-09-10T18:35:42.221602+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[18597]: audit 2024-09-10T18:35:42.231468+0000 mon.a (mon.0) 794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:35:43.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[31607]: cluster 2024-09-10T18:35:41.236708+0000 mgr.y (mgr.24512) 43 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:43.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[31607]: audit 2024-09-10T18:35:41.892625+0000 mgr.y (mgr.24512) 44 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:43.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[31607]: audit 2024-09-10T18:35:42.207097+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.151:0/419938161' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:43.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[31607]: audit 2024-09-10T18:35:42.221602+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:35:43.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:42 smithi100 bash[31607]: audit 2024-09-10T18:35:42.231468+0000 mon.a (mon.0) 794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:35:44.813 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-10T18:35:44.813 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T18:35:44.813 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T18:35:44.814 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:35:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:35:44.832 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T18:35:44.832 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T18:35:44.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T18:35:45.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:44 smithi151 bash[22565]: cluster 2024-09-10T18:35:43.237430+0000 mgr.y (mgr.24512) 45 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:44 smithi100 bash[18597]: cluster 2024-09-10T18:35:43.237430+0000 mgr.y (mgr.24512) 45 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:45.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:44 smithi100 bash[31607]: cluster 2024-09-10T18:35:43.237430+0000 mgr.y (mgr.24512) 45 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:47.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:46 smithi151 bash[22565]: cluster 2024-09-10T18:35:45.238790+0000 mgr.y (mgr.24512) 46 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:46 smithi100 bash[18597]: cluster 2024-09-10T18:35:45.238790+0000 mgr.y (mgr.24512) 46 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:47.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:46 smithi100 bash[31607]: cluster 2024-09-10T18:35:45.238790+0000 mgr.y (mgr.24512) 46 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:49.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:48 smithi151 bash[22565]: cluster 2024-09-10T18:35:47.239512+0000 mgr.y (mgr.24512) 47 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:49.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:48 smithi100 bash[18597]: cluster 2024-09-10T18:35:47.239512+0000 mgr.y (mgr.24512) 47 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:49.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:48 smithi100 bash[31607]: cluster 2024-09-10T18:35:47.239512+0000 mgr.y (mgr.24512) 47 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:49.610 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:50 smithi100 bash[18597]: cluster 2024-09-10T18:35:49.240241+0000 mgr.y (mgr.24512) 48 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:51.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:50 smithi100 bash[31607]: cluster 2024-09-10T18:35:49.240241+0000 mgr.y (mgr.24512) 48 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:51.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:50 smithi151 bash[22565]: cluster 2024-09-10T18:35:49.240241+0000 mgr.y (mgr.24512) 48 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:52.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:35:51 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:35:52.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.989+0000 7f5cc89de640 1 -- 172.21.15.100:0/2835456890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fb950 msgr2=0x7f5cc40fbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:52.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.989+0000 7f5cc89de640 1 --2- 172.21.15.100:0/2835456890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fb950 0x7f5cc40fbdc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004980 tx=0x7f5cac009da0 comp rx=0 tx=0).stop 2024-09-10T18:35:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- 172.21.15.100:0/2835456890 shutdown_connections 2024-09-10T18:35:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 --2- 172.21.15.100:0/2835456890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc40fc300 0x7f5cc4108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 --2- 172.21.15.100:0/2835456890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fb950 0x7f5cc40fbdc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 --2- 172.21.15.100:0/2835456890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc4101760 0x7f5cc4101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- 172.21.15.100:0/2835456890 >> 172.21.15.100:0/2835456890 conn(0x7f5cc4072440 msgr2=0x7f5cc4072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- 172.21.15.100:0/2835456890 shutdown_connections 2024-09-10T18:35:52.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- 172.21.15.100:0/2835456890 wait complete. 2024-09-10T18:35:52.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 Processor -- start 2024-09-10T18:35:52.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- start start 2024-09-10T18:35:52.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc40fb950 0x7f5cc4114fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:52.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fc300 0x7f5cc41154e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc4101760 0x7f5cc410f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc410f780 con 0x7f5cc40fc300 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc410f8f0 con 0x7f5cc4101760 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc410fab0 con 0x7f5cc40fb950 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fc300 0x7f5cc41154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc40fb950 0x7f5cc4114fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fc300 0x7f5cc41154e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33954/0 (socket says 172.21.15.100:33954) 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc1d9b640 1 -- 172.21.15.100:0/3541929891 learned_addr learned my addr 172.21.15.100:0/3541929891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc2d9d640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc4101760 0x7f5cc410f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc259c640 1 -- 172.21.15.100:0/3541929891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc4101760 msgr2=0x7f5cc410f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc259c640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc4101760 0x7f5cc410f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc259c640 1 -- 172.21.15.100:0/3541929891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fc300 msgr2=0x7f5cc41154e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:52.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc259c640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fc300 0x7f5cc41154e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:52.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc259c640 1 -- 172.21.15.100:0/3541929891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac005040 con 0x7f5cc40fb950 2024-09-10T18:35:52.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc259c640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc40fb950 0x7f5cc4114fa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4002820 tx=0x7f5cb4009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:52.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cbaffd640 1 -- 172.21.15.100:0/3541929891 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4007c30 con 0x7f5cc40fb950 2024-09-10T18:35:52.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cbaffd640 1 -- 172.21.15.100:0/3541929891 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f5cb400f040 con 0x7f5cc40fb950 2024-09-10T18:35:52.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc410fd90 con 0x7f5cc40fb950 2024-09-10T18:35:52.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cbaffd640 1 -- 172.21.15.100:0/3541929891 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4009340 con 0x7f5cc40fb950 2024-09-10T18:35:52.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.993+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc411bf90 con 0x7f5cc40fb950 2024-09-10T18:35:52.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.997+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc40fce80 con 0x7f5cc40fb950 2024-09-10T18:35:52.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.997+0000 7f5cbaffd640 1 -- 172.21.15.100:0/3541929891 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f5cb4009770 con 0x7f5cc40fb950 2024-09-10T18:35:52.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.997+0000 7f5cbaffd640 1 --2- 172.21.15.100:0/3541929891 >> v2:172.21.15.100:6800/2155730476 conn(0x7f5cb006eec0 0x7f5cb0071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:35:52.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.997+0000 7f5cc1d9b640 1 --2- 172.21.15.100:0/3541929891 >> v2:172.21.15.100:6800/2155730476 conn(0x7f5cb006eec0 0x7f5cb0071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:35:52.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.997+0000 7f5cbaffd640 1 -- 172.21.15.100:0/3541929891 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6168+0+0 (secure 0 0 0) 0x7f5cb408e300 con 0x7f5cc40fb950 2024-09-10T18:35:52.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:52.997+0000 7f5cc1d9b640 1 --2- 172.21.15.100:0/3541929891 >> v2:172.21.15.100:6800/2155730476 conn(0x7f5cb006eec0 0x7f5cb0071370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4110860 tx=0x7f5cac009da0 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:35:53.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.001+0000 7f5cbaffd640 1 -- 172.21.15.100:0/3541929891 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5cb405b280 con 0x7f5cc40fb950 2024-09-10T18:35:53.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.221+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5cc4110610 con 0x7f5cc40fb950 2024-09-10T18:35:53.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.225+0000 7f5cbaffd640 1 -- 172.21.15.100:0/3541929891 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) v1 ==== 74+0+176631 (secure 0 0 0) 0x7f5cb40030b0 con 0x7f5cc40fb950 2024-09-10T18:35:53.226 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:35:53.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 >> v2:172.21.15.100:6800/2155730476 conn(0x7f5cb006eec0 msgr2=0x7f5cb0071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:53.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 --2- 172.21.15.100:0/3541929891 >> v2:172.21.15.100:6800/2155730476 conn(0x7f5cb006eec0 0x7f5cb0071370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4110860 tx=0x7f5cac009da0 comp rx=0 tx=0).stop 2024-09-10T18:35:53.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc40fb950 msgr2=0x7f5cc4114fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:53.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc40fb950 0x7f5cc4114fa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4002820 tx=0x7f5cb4009f30 comp rx=0 tx=0).stop 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 shutdown_connections 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 --2- 172.21.15.100:0/3541929891 >> v2:172.21.15.100:6800/2155730476 conn(0x7f5cb006eec0 0x7f5cb0071370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5cc4101760 0x7f5cc410f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cc40fc300 0x7f5cc41154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 --2- 172.21.15.100:0/3541929891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cc40fb950 0x7f5cc4114fa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 >> 172.21.15.100:0/3541929891 conn(0x7f5cc4072440 msgr2=0x7f5cc40fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 shutdown_connections 2024-09-10T18:35:53.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:35:53.229+0000 7f5cc89de640 1 -- 172.21.15.100:0/3541929891 wait complete. 2024-09-10T18:35:53.260 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:52 smithi100 bash[18597]: cluster 2024-09-10T18:35:51.241735+0000 mgr.y (mgr.24512) 49 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:53.260 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:52 smithi100 bash[18597]: audit 2024-09-10T18:35:51.903975+0000 mgr.y (mgr.24512) 50 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:53.260 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:52 smithi100 bash[31607]: cluster 2024-09-10T18:35:51.241735+0000 mgr.y (mgr.24512) 49 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:53.260 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:52 smithi100 bash[31607]: audit 2024-09-10T18:35:51.903975+0000 mgr.y (mgr.24512) 50 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:53.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:52 smithi151 bash[22565]: cluster 2024-09-10T18:35:51.241735+0000 mgr.y (mgr.24512) 49 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:53.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:52 smithi151 bash[22565]: audit 2024-09-10T18:35:51.903975+0000 mgr.y (mgr.24512) 50 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:35:53.950 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":21,"flags":0,"active_gid":24512,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":2155730476}]},"active_addr":"172.21.15.100:6800/2155730476","active_change":"2024-09-10T18:35:09.193567+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24511,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.100:8443/","prometheus":"http://172.21.15.100:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":65,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1006566135}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":109898964}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3790539326}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1074084133}]}]} 2024-09-10T18:35:53.955 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T18:35:53.955 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T18:35:53.955 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:35:54.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:53 smithi100 bash[31607]: audit 2024-09-10T18:35:53.222751+0000 mon.c (mon.2) 54 : audit [DBG] from='client.? 172.21.15.100:0/3541929891' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:35:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:53 smithi100 bash[18597]: audit 2024-09-10T18:35:53.222751+0000 mon.c (mon.2) 54 : audit [DBG] from='client.? 172.21.15.100:0/3541929891' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:35:54.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:53 smithi151 bash[22565]: audit 2024-09-10T18:35:53.222751+0000 mon.c (mon.2) 54 : audit [DBG] from='client.? 172.21.15.100:0/3541929891' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:35:54.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:35:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:35:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:35:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:54 smithi100 bash[18597]: cluster 2024-09-10T18:35:53.242222+0000 mgr.y (mgr.24512) 51 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:54 smithi100 bash[18597]: audit 2024-09-10T18:35:54.314507+0000 mon.c (mon.2) 55 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:54 smithi100 bash[31607]: cluster 2024-09-10T18:35:53.242222+0000 mgr.y (mgr.24512) 51 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:54 smithi100 bash[31607]: audit 2024-09-10T18:35:54.314507+0000 mon.c (mon.2) 55 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:55.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:54 smithi151 bash[22565]: cluster 2024-09-10T18:35:53.242222+0000 mgr.y (mgr.24512) 51 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:55.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:54 smithi151 bash[22565]: audit 2024-09-10T18:35:54.314507+0000 mon.c (mon.2) 55 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:35:57.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:56 smithi100 bash[18597]: cluster 2024-09-10T18:35:55.243694+0000 mgr.y (mgr.24512) 52 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:56 smithi100 bash[31607]: cluster 2024-09-10T18:35:55.243694+0000 mgr.y (mgr.24512) 52 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:57.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:56 smithi151 bash[22565]: cluster 2024-09-10T18:35:55.243694+0000 mgr.y (mgr.24512) 52 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:35:58.721 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:35:58 smithi100 bash[18597]: cluster 2024-09-10T18:35:57.244533+0000 mgr.y (mgr.24512) 53 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:35:58 smithi100 bash[31607]: cluster 2024-09-10T18:35:57.244533+0000 mgr.y (mgr.24512) 53 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:35:59.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:35:58 smithi151 bash[22565]: cluster 2024-09-10T18:35:57.244533+0000 mgr.y (mgr.24512) 53 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:00 smithi151 bash[22565]: cluster 2024-09-10T18:35:59.245285+0000 mgr.y (mgr.24512) 54 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:00 smithi151 bash[22565]: audit 2024-09-10T18:36:00.900603+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:00 smithi151 bash[22565]: audit 2024-09-10T18:36:00.912902+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:00 smithi151 bash[22565]: audit 2024-09-10T18:36:00.921989+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:00 smithi151 bash[22565]: audit 2024-09-10T18:36:00.929269+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:00 smithi151 systemd[1]: Started Ceph grafana.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:36:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[18597]: cluster 2024-09-10T18:35:59.245285+0000 mgr.y (mgr.24512) 54 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[18597]: audit 2024-09-10T18:36:00.900603+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[18597]: audit 2024-09-10T18:36:00.912902+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[18597]: audit 2024-09-10T18:36:00.921989+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[18597]: audit 2024-09-10T18:36:00.929269+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[31607]: cluster 2024-09-10T18:35:59.245285+0000 mgr.y (mgr.24512) 54 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[31607]: audit 2024-09-10T18:36:00.900603+0000 mon.a (mon.0) 795 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[31607]: audit 2024-09-10T18:36:00.912902+0000 mon.a (mon.0) 796 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[31607]: audit 2024-09-10T18:36:00.921989+0000 mon.a (mon.0) 797 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:00 smithi100 bash[31607]: audit 2024-09-10T18:36:00.929269+0000 mon.a (mon.0) 798 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 -- 172.21.15.100:0/1460418158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 msgr2=0x7faccc109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 --2- 172.21.15.100:0/1460418158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc109600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7facbc009cd0 tx=0x7facbc004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 -- 172.21.15.100:0/1460418158 shutdown_connections 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 --2- 172.21.15.100:0/1460418158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc109600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 --2- 172.21.15.100:0/1460418158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faccc065fd0 0x7faccc066440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 --2- 172.21.15.100:0/1460418158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faccc102850 0x7faccc102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 -- 172.21.15.100:0/1460418158 >> 172.21.15.100:0/1460418158 conn(0x7faccc0724b0 msgr2=0x7faccc0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 -- 172.21.15.100:0/1460418158 shutdown_connections 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 -- 172.21.15.100:0/1460418158 wait complete. 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 Processor -- start 2024-09-10T18:36:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 -- start start 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faccc065fd0 0x7faccc114da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.969+0000 7facd0fea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc1152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facd0fea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faccc102850 0x7faccc10ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facd0fea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc10f550 con 0x7faccc066980 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facd0fea640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc10f6c0 con 0x7faccc102850 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facd0fea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc10f880 con 0x7faccc065fd0 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc1152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc1152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58886/0 (socket says 172.21.15.100:58886) 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 -- 172.21.15.100:0/556227730 learned_addr learned my addr 172.21.15.100:0/556227730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facca59c640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faccc065fd0 0x7faccc114da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7faccad9d640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faccc102850 0x7faccc10ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 -- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faccc065fd0 msgr2=0x7faccc114da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faccc065fd0 0x7faccc114da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 -- 172.21.15.100:0/556227730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faccc102850 msgr2=0x7faccc10ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faccc102850 0x7faccc10ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 -- 172.21.15.100:0/556227730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0005040 con 0x7faccc066980 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facca59c640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faccc065fd0 0x7faccc114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facc9d9b640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc1152e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7facc0009780 tx=0x7facc0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facbaffd640 1 -- 172.21.15.100:0/556227730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00099c0 con 0x7faccc066980 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facbaffd640 1 -- 172.21.15.100:0/556227730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7facc000f040 con 0x7faccc066980 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facbaffd640 1 -- 172.21.15.100:0/556227730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0013c30 con 0x7faccc066980 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc0098b0 con 0x7faccc066980 2024-09-10T18:36:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc11bbc0 con 0x7faccc066980 2024-09-10T18:36:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.973+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc0fdf70 con 0x7faccc066980 2024-09-10T18:36:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.977+0000 7facbaffd640 1 -- 172.21.15.100:0/556227730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7facc000eb30 con 0x7faccc066980 2024-09-10T18:36:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.977+0000 7facbaffd640 1 --2- 172.21.15.100:0/556227730 >> v2:172.21.15.100:6800/2155730476 conn(0x7facb406eec0 0x7facb4071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.977+0000 7facca59c640 1 --2- 172.21.15.100:0/556227730 >> v2:172.21.15.100:6800/2155730476 conn(0x7facb406eec0 0x7facb4071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.977+0000 7facbaffd640 1 -- 172.21.15.100:0/556227730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6168+0+0 (secure 0 0 0) 0x7facc008e290 con 0x7faccc066980 2024-09-10T18:36:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.977+0000 7facca59c640 1 --2- 172.21.15.100:0/556227730 >> v2:172.21.15.100:6800/2155730476 conn(0x7facb406eec0 0x7facb4071370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7facc4004fd0 tx=0x7facc400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:01.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:01.981+0000 7facbaffd640 1 -- 172.21.15.100:0/556227730 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7facc005b220 con 0x7faccc066980 2024-09-10T18:36:02.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.157+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faccc1103e0 con 0x7faccc066980 2024-09-10T18:36:02.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.161+0000 7facbaffd640 1 -- 172.21.15.100:0/556227730 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+19258 (secure 0 0 0) 0x7facc005abc0 con 0x7faccc066980 2024-09-10T18:36:02.162 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:36:02.162 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":65,"fsid":"505ebbde-6fa1-11ef-bcea-c7b262605968","created":"2024-09-10T18:23:17.238038+0000","modified":"2024-09-10T18:35:09.193138+0000","last_up_change":"2024-09-10T18:33:26.598092+0000","last_in_change":"2024-09-10T18:32:58.920619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:28:49.721821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:34:09.153206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:34:10.904827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:34:12.858132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"datapool","create_time":"2024-09-10T18:34:14.149048+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:34:15.315889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"9a69cb5a-7a42-4fb4-861d-55a459379aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":2899184110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2899184110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2899184110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":2899184110}]},"public_addr":"172.21.15.100:6801/2899184110","cluster_addr":"172.21.15.100:6802/2899184110","heartbeat_back_addr":"172.21.15.100:6804/2899184110","heartbeat_front_addr":"172.21.15.100:6803/2899184110","state":["exists","up"]},{"osd":1,"uuid":"b6ad7728-dc92-4291-99cd-18ebfacd6c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6805","nonce":59055423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":59055423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":59055423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":59055423}]},"public_addr":"172.21.15.100:6805/59055423","cluster_addr":"172.21.15.100:6806/59055423","heartbeat_back_addr":"172.21.15.100:6808/59055423","heartbeat_front_addr":"172.21.15.100:6807/59055423","state":["exists","up"]},{"osd":2,"uuid":"ce92944f-0b92-46c8-a6b7-6cdd00f9736a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":370401307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":370401307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":370401307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6811","nonce":370401307}]},"public_addr":"172.21.15.100:6809/370401307","cluster_addr":"172.21.15.100:6810/370401307","heartbeat_back_addr":"172.21.15.100:6812/370401307","heartbeat_front_addr":"172.21.15.100:6811/370401307","state":["exists","up"]},{"osd":3,"uuid":"8340f7e5-8d39-4316-ae1e-809bcf286e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":2754318461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2754318461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2754318461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":2754318461}]},"public_addr":"172.21.15.100:6813/2754318461","cluster_addr":"172.21.15.100:6814/2754318461","heartbeat_back_addr":"172.21.15.100:6816/2754318461","heartbeat_front_addr":"172.21.15.100:6815/2754318461","state":["exists","up"]},{"osd":4,"uuid":"80851ca1-0c33-4ac6-80b5-92bbcaa3dacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2989412742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2989412742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6803","nonce":2989412742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2989412742}]},"public_addr":"172.21.15.151:6800/2989412742","cluster_addr":"172.21.15.151:6801/2989412742","heartbeat_back_addr":"172.21.15.151:6803/2989412742","heartbeat_front_addr":"172.21.15.151:6802/2989412742","state":["exists","up"]},{"osd":5,"uuid":"115d5a71-f0c1-4d54-8cb7-7e190a5405a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3330203218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":3330203218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6807","nonce":3330203218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3330203218}]},"public_addr":"172.21.15.151:6804/3330203218","cluster_addr":"172.21.15.151:6805/3330203218","heartbeat_back_addr":"172.21.15.151:6807/3330203218","heartbeat_front_addr":"172.21.15.151:6806/3330203218","state":["exists","up"]},{"osd":6,"uuid":"3dbae406-f3c5-43e0-98bf-9b4d26715d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":251427555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6809","nonce":251427555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":251427555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":251427555}]},"public_addr":"172.21.15.151:6808/251427555","cluster_addr":"172.21.15.151:6809/251427555","heartbeat_back_addr":"172.21.15.151:6811/251427555","heartbeat_front_addr":"172.21.15.151:6810/251427555","state":["exists","up"]},{"osd":7,"uuid":"df3a51c0-0655-462f-b528-e4ccd5d130ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4021034041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6813","nonce":4021034041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6815","nonce":4021034041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4021034041}]},"public_addr":"172.21.15.151:6812/4021034041","cluster_addr":"172.21.15.151:6813/4021034041","heartbeat_back_addr":"172.21.15.151:6815/4021034041","heartbeat_front_addr":"172.21.15.151:6814/4021034041","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:26:57.052592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:27:51.521280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:28:46.335623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:29:42.412430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:30:36.277604+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:31:31.151158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:32:28.116557+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:33:25.354725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/2483032053":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/1686873674":"2024-09-11T18:23:45.175932+0000","172.21.15.100:0/4071543765":"2024-09-11T18:23:45.175932+0000","172.21.15.100:6800/1728522861":"2024-09-11T18:23:45.175932+0000","172.21.15.100:0/1105054798":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/69624316":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/41752804":"2024-09-11T18:23:45.175932+0000","172.21.15.100:6800/2893280940":"2024-09-11T18:24:19.124827+0000","172.21.15.100:6800/3986579393":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/2285226173":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/124948575":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/1013137826":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/1969858017":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/2038780913":"2024-09-11T18:35:09.193020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:36:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.161+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 >> v2:172.21.15.100:6800/2155730476 conn(0x7facb406eec0 msgr2=0x7facb4071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.161+0000 7facd0fea640 1 --2- 172.21.15.100:0/556227730 >> v2:172.21.15.100:6800/2155730476 conn(0x7facb406eec0 0x7facb4071370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7facc4004fd0 tx=0x7facc400b040 comp rx=0 tx=0).stop 2024-09-10T18:36:02.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.161+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 msgr2=0x7faccc1152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:02.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.161+0000 7facd0fea640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc1152e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7facc0009780 tx=0x7facc0009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:02.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 shutdown_connections 2024-09-10T18:36:02.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 --2- 172.21.15.100:0/556227730 >> v2:172.21.15.100:6800/2155730476 conn(0x7facb406eec0 0x7facb4071370 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:02.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faccc102850 0x7faccc10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faccc066980 0x7faccc1152e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 --2- 172.21.15.100:0/556227730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faccc065fd0 0x7faccc114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 >> 172.21.15.100:0/556227730 conn(0x7faccc0724b0 msgr2=0x7faccc0fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 shutdown_connections 2024-09-10T18:36:02.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:02.165+0000 7facd0fea640 1 -- 172.21.15.100:0/556227730 wait complete. 2024-09-10T18:36:02.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:36:01 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:36:02.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:01 smithi151 bash[22565]: audit 2024-09-10T18:36:00.975355+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:02.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:01 smithi151 bash[22565]: cluster 2024-09-10T18:36:01.246663+0000 mgr.y (mgr.24512) 55 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:02.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:01 smithi151 bash[22565]: audit 2024-09-10T18:36:01.915432+0000 mgr.y (mgr.24512) 56 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:02.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:01 smithi100 bash[18597]: audit 2024-09-10T18:36:00.975355+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:02.189 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:01 smithi100 bash[18597]: cluster 2024-09-10T18:36:01.246663+0000 mgr.y (mgr.24512) 55 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:02.190 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:01 smithi100 bash[18597]: audit 2024-09-10T18:36:01.915432+0000 mgr.y (mgr.24512) 56 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:02.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:01 smithi100 bash[31607]: audit 2024-09-10T18:36:00.975355+0000 mon.c (mon.2) 56 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:02.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:01 smithi100 bash[31607]: cluster 2024-09-10T18:36:01.246663+0000 mgr.y (mgr.24512) 55 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:02.190 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:01 smithi100 bash[31607]: audit 2024-09-10T18:36:01.915432+0000 mgr.y (mgr.24512) 56 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:02.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261537103Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T18:36:02.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261737225Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T18:36:02.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261762417Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T18:36:02.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261771542Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T18:36:02.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261779257Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T18:36:02.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261787558Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T18:36:02.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261794687Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261812912Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261850687Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261862595Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261873576Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261885236Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261897686Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=settings t=2024-09-10T18:36:02.261909553Z level=info msg="App mode production" 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=sqlstore t=2024-09-10T18:36:02.261985889Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=sqlstore t=2024-09-10T18:36:02.26201953Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.263379695Z level=info msg="Starting DB migrations" 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.263471676Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.292607233Z level=info msg="Executing migration" id="create user table" 2024-09-10T18:36:02.531 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.337948977Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T18:36:02.532 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.373278991Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T18:36:02.532 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.415390618Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T18:36:02.532 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.479602538Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T18:36:02.532 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.516333498Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T18:36:02.801 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.53848616Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T18:36:02.801 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.580547324Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T18:36:02.802 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.605959857Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T18:36:02.802 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.648127496Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T18:36:02.802 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.673300659Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T18:36:02.802 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.711909295Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T18:36:02.802 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.73766181Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T18:36:02.802 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.762911956Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T18:36:02.802 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.788308363Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T18:36:02.906 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T18:36:02.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:36:03.056 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[22565]: audit 2024-09-10T18:36:02.161110+0000 mon.a (mon.0) 799 : audit [DBG] from='client.? 172.21.15.100:0/556227730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:36:03.056 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.830359963Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T18:36:03.056 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.852198853Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T18:36:03.056 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.877546741Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T18:36:03.056 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.902816577Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T18:36:03.056 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.944996959Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T18:36:03.056 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:02 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:02.975642108Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T18:36:03.056 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.04282995Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T18:36:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:02 smithi100 bash[18597]: audit 2024-09-10T18:36:02.161110+0000 mon.a (mon.0) 799 : audit [DBG] from='client.? 172.21.15.100:0/556227730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:36:03.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:02 smithi100 bash[31607]: audit 2024-09-10T18:36:02.161110+0000 mon.a (mon.0) 799 : audit [DBG] from='client.? 172.21.15.100:0/556227730' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:36:03.313 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.068191159Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T18:36:03.313 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.110165143Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T18:36:03.313 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.135595631Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T18:36:03.313 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.173372571Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T18:36:03.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.198773652Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T18:36:03.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.224052462Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T18:36:03.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.249348036Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T18:36:03.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.274764031Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T18:36:03.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.300147993Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T18:36:03.590 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.34213935Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T18:36:03.590 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.375879345Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T18:36:03.590 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.404022496Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T18:36:03.591 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.439970117Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T18:36:03.591 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.482027157Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T18:36:03.591 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.507330537Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T18:36:03.591 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.577096203Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T18:36:03.864 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.610467859Z level=info msg="Executing migration" id="create star table" 2024-09-10T18:36:03.864 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.652447086Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T18:36:03.864 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.677903518Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T18:36:03.864 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.722760168Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T18:36:03.864 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.766277658Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T18:36:03.864 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.808567719Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T18:36:03.864 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.85068474Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[22565]: cluster 2024-09-10T18:36:03.247447+0000 mgr.y (mgr.24512) 57 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.875992509Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.901308066Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.943339096Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:03 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:03.987927746Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.005017813Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.033051595Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.058421662Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T18:36:04.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.133759457Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T18:36:04.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:03 smithi100 bash[31607]: cluster 2024-09-10T18:36:03.247447+0000 mgr.y (mgr.24512) 57 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:03 smithi100 bash[18597]: cluster 2024-09-10T18:36:03.247447+0000 mgr.y (mgr.24512) 57 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:04.405 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.161524496Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T18:36:04.405 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.186801887Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T18:36:04.405 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.212105488Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T18:36:04.405 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.239741813Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T18:36:04.405 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.265131079Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T18:36:04.405 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.290545192Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T18:36:04.405 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.335590269Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T18:36:04.406 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.364070048Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T18:36:04.406 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.391970696Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T18:36:04.676 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:36:04.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.434088389Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T18:36:04.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.478541584Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T18:36:04.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.503896567Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T18:36:04.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.52929382Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T18:36:04.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.565395715Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T18:36:04.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.607422961Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T18:36:04.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.679260953Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T18:36:04.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.704677522Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T18:36:04.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.746672001Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T18:36:04.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.772057754Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T18:36:04.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.797444956Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T18:36:04.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.83949372Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T18:36:04.968 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.861275138Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T18:36:04.968 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.886586819Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T18:36:04.968 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.911950829Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T18:36:04.968 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.954088564Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:04 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:04.979426788Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.004689962Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.04315713Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.068458302Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.093810913Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.119080149Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.161228228Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T18:36:05.250 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.236763217Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T18:36:05.520 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[22565]: audit 2024-09-10T18:36:04.329334+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:05.520 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.258415413Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T18:36:05.520 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.28386304Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T18:36:05.520 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.325946954Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T18:36:05.520 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.368251029Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T18:36:05.521 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.393575979Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T18:36:05.521 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.439516287Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T18:36:05.521 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.481673228Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T18:36:05.521 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.50700702Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T18:36:05.777 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.532406278Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T18:36:05.777 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.55765818Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T18:36:05.777 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.599788389Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T18:36:05.777 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.641794943Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T18:36:05.778 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.688565409Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T18:36:05.778 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.764026239Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T18:36:05.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:05 smithi100 bash[18597]: audit 2024-09-10T18:36:04.329334+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:05.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:05 smithi100 bash[31607]: audit 2024-09-10T18:36:04.329334+0000 mon.a (mon.0) 800 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:06.052 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.789638407Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T18:36:06.052 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.823154545Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T18:36:06.052 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.865169701Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T18:36:06.052 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.890789728Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T18:36:06.053 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.92928277Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T18:36:06.053 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.971312594Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T18:36:06.053 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:05 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:05.996538812Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T18:36:06.053 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.01355794Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T18:36:06.053 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.038978885Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T18:36:06.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.062191889Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T18:36:06.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.087450847Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T18:36:06.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.112720577Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T18:36:06.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.169508413Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T18:36:06.336 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.194858265Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T18:36:06.336 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.224476947Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T18:36:06.336 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.249776514Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T18:36:06.545 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[22565]: cluster 2024-09-10T18:36:05.248920+0000 mgr.y (mgr.24512) 58 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:06.546 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.333727492Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T18:36:06.546 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.359127002Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T18:36:06.546 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.384436156Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T18:36:06.546 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.410033287Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T18:36:06.546 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.464701071Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T18:36:06.546 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.5068705Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T18:36:06.587 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.532274633Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T18:36:06.588 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.574433234Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T18:36:06.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:06 smithi100 bash[18597]: cluster 2024-09-10T18:36:05.248920+0000 mgr.y (mgr.24512) 58 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:06.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:06 smithi100 bash[31607]: cluster 2024-09-10T18:36:05.248920+0000 mgr.y (mgr.24512) 58 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:06.889 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.599803927Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T18:36:06.889 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.638092803Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T18:36:06.889 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.680312495Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T18:36:06.889 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.705556514Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T18:36:06.889 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.747691094Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T18:36:06.889 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.789916211Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T18:36:06.889 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.874974328Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T18:36:07.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.900110047Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T18:36:07.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.944706345Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T18:36:07.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:06 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:06.986871626Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T18:36:07.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.012222619Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T18:36:07.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.037471432Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T18:36:07.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.069513451Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T18:36:07.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.103389409Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T18:36:07.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.131533709Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T18:36:07.425 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.18670916Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T18:36:07.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.211997145Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T18:36:07.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.22901267Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T18:36:07.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.271142558Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T18:36:07.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.313274934Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T18:36:07.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.351322588Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T18:36:07.452 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:07.696 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.426962716Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.452441241Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.494403706Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.519775699Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.545467356Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.590498179Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.615754925Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.657782912Z level=info msg="Executing migration" id="create session table" 2024-09-10T18:36:07.697 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.683249344Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T18:36:08.012 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.708453996Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.733816594Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.776092047Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.801406095Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.826773562Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.852130107Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.873660836Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.915741589Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T18:36:08.013 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:07 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:07.999540109Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T18:36:08.268 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: audit 2024-09-10T18:36:07.152142+0000 mon.a (mon.0) 801 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.268 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: audit 2024-09-10T18:36:07.161011+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.268 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: cluster 2024-09-10T18:36:07.249637+0000 mgr.y (mgr.24512) 59 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:08.268 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: audit 2024-09-10T18:36:07.347282+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.268 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: audit 2024-09-10T18:36:07.358538+0000 mon.a (mon.0) 804 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.268 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: audit 2024-09-10T18:36:07.360872+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:08.269 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: audit 2024-09-10T18:36:07.362399+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:08.269 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: audit 2024-09-10T18:36:07.371570+0000 mon.a (mon.0) 805 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.269 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: cephadm 2024-09-10T18:36:07.426303+0000 mgr.y (mgr.24512) 60 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:36:08.269 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[22565]: cephadm 2024-09-10T18:36:07.431792+0000 mgr.y (mgr.24512) 61 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi100 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.041561006Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.066852985Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.09225424Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.114537194Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.156650862Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.182077186Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.207469695Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T18:36:08.269 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.23286029Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T18:36:08.270 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.255058004Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T18:36:08.536 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.280435669Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T18:36:08.537 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.322480652Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T18:36:08.537 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.364665957Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T18:36:08.537 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.388814886Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T18:36:08.537 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.414150645Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T18:36:08.537 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.456123946Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T18:36:08.537 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.523319871Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: audit 2024-09-10T18:36:07.152142+0000 mon.a (mon.0) 801 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: audit 2024-09-10T18:36:07.161011+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: cluster 2024-09-10T18:36:07.249637+0000 mgr.y (mgr.24512) 59 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: audit 2024-09-10T18:36:07.347282+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: audit 2024-09-10T18:36:07.358538+0000 mon.a (mon.0) 804 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: audit 2024-09-10T18:36:07.360872+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: audit 2024-09-10T18:36:07.362399+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:08.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: audit 2024-09-10T18:36:07.371570+0000 mon.a (mon.0) 805 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: cephadm 2024-09-10T18:36:07.426303+0000 mgr.y (mgr.24512) 60 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[18597]: cephadm 2024-09-10T18:36:07.431792+0000 mgr.y (mgr.24512) 61 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi100 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: audit 2024-09-10T18:36:07.152142+0000 mon.a (mon.0) 801 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: audit 2024-09-10T18:36:07.161011+0000 mon.a (mon.0) 802 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: cluster 2024-09-10T18:36:07.249637+0000 mgr.y (mgr.24512) 59 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: audit 2024-09-10T18:36:07.347282+0000 mon.a (mon.0) 803 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: audit 2024-09-10T18:36:07.358538+0000 mon.a (mon.0) 804 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: audit 2024-09-10T18:36:07.360872+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: audit 2024-09-10T18:36:07.362399+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: audit 2024-09-10T18:36:07.371570+0000 mon.a (mon.0) 805 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: cephadm 2024-09-10T18:36:07.426303+0000 mgr.y (mgr.24512) 60 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:36:08.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:08 smithi100 bash[31607]: cephadm 2024-09-10T18:36:07.431792+0000 mgr.y (mgr.24512) 61 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi100 2024-09-10T18:36:08.823 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.548737463Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.590770473Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.63275389Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.65827043Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.683635501Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.708942687Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.73431764Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.759703928Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T18:36:08.824 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.81028085Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T18:36:09.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.835735924Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T18:36:09.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.877763608Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T18:36:09.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.903138226Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T18:36:09.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.935508113Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T18:36:09.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:08 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:08.960897358Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T18:36:09.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.002826863Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T18:36:09.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.069873606Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T18:36:09.353 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.095406633Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.120749333Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.162758786Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.188362663Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.213713299Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.230758818Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.256017596Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.297935423Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.340370482Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T18:36:09.354 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.237425+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2024-09-10T18:36:09.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.238036+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.238530+0000 mon.a (mon.0) 806 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2024-09-10T18:36:09.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.238819+0000 mon.c (mon.2) 61 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.239562+0000 mon.c (mon.2) 62 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-10T18:36:09.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.239815+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.240640+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[22565]: audit 2024-09-10T18:36:09.241116+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.237425+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.238036+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.238530+0000 mon.a (mon.0) 806 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.238819+0000 mon.c (mon.2) 61 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.239562+0000 mon.c (mon.2) 62 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.239815+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.240640+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.603 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[31607]: audit 2024-09-10T18:36:09.241116+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.237425+0000 mon.c (mon.2) 59 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.238036+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.238530+0000 mon.a (mon.0) 806 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.238819+0000 mon.c (mon.2) 61 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.239562+0000 mon.c (mon.2) 62 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.239815+0000 mon.a (mon.0) 807 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.240640+0000 mon.a (mon.0) 808 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-10T18:36:09.604 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:09 smithi100 bash[18597]: audit 2024-09-10T18:36:09.241116+0000 mon.a (mon.0) 809 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-10T18:36:09.627 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.382518359Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T18:36:09.628 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.424597126Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T18:36:09.628 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.449946493Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T18:36:09.628 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.491955569Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T18:36:09.628 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.513577651Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T18:36:09.628 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.538850855Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T18:36:09.628 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.614395071Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T18:36:09.893 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.639841289Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T18:36:09.893 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.66510182Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T18:36:09.893 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.690415958Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T18:36:09.894 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.711701534Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T18:36:09.894 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.762151697Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T18:36:09.894 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.80440041Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T18:36:09.894 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.829655678Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T18:36:09.894 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.880222704Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.905610004Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.935292641Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:09 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:09.977399046Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.00272629Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.028064475Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.053360178Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.128194366Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T18:36:10.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.162493499Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T18:36:10.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.192776062Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T18:36:10.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.234904394Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T18:36:10.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.2603238Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T18:36:10.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.302259585Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T18:36:10.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.350559356Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T18:36:10.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.375824656Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: cluster 2024-09-10T18:36:09.250278+0000 mgr.y (mgr.24512) 62 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: audit 2024-09-10T18:36:09.298370+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]': finished 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: audit 2024-09-10T18:36:09.298666+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: audit 2024-09-10T18:36:09.298917+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: audit 2024-09-10T18:36:09.299164+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: cluster 2024-09-10T18:36:09.299390+0000 mon.a (mon.0) 814 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: audit 2024-09-10T18:36:09.313887+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[22565]: cluster 2024-09-10T18:36:10.302503+0000 mon.a (mon.0) 815 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:36:10.714 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.418072773Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T18:36:10.714 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.457685051Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T18:36:10.714 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.499760774Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T18:36:10.714 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.532950367Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T18:36:10.714 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.574962827Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T18:36:10.714 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.617313352Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T18:36:10.714 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.700940972Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T18:36:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: cluster 2024-09-10T18:36:09.250278+0000 mgr.y (mgr.24512) 62 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: audit 2024-09-10T18:36:09.298370+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]': finished 2024-09-10T18:36:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: audit 2024-09-10T18:36:09.298666+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-10T18:36:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: audit 2024-09-10T18:36:09.298917+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-10T18:36:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: audit 2024-09-10T18:36:09.299164+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: cluster 2024-09-10T18:36:09.299390+0000 mon.a (mon.0) 814 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: audit 2024-09-10T18:36:09.313887+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[18597]: cluster 2024-09-10T18:36:10.302503+0000 mon.a (mon.0) 815 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: cluster 2024-09-10T18:36:09.250278+0000 mgr.y (mgr.24512) 62 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: audit 2024-09-10T18:36:09.298370+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [7, 2]}]': finished 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: audit 2024-09-10T18:36:09.298666+0000 mon.a (mon.0) 811 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: audit 2024-09-10T18:36:09.298917+0000 mon.a (mon.0) 812 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: audit 2024-09-10T18:36:09.299164+0000 mon.a (mon.0) 813 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: cluster 2024-09-10T18:36:09.299390+0000 mon.a (mon.0) 814 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: audit 2024-09-10T18:36:09.313887+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:10 smithi100 bash[31607]: cluster 2024-09-10T18:36:10.302503+0000 mon.a (mon.0) 815 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:36:10.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.743169987Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T18:36:10.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.768655123Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T18:36:10.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.793882363Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T18:36:10.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.819217067Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T18:36:10.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.844662368Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T18:36:10.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.869876878Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T18:36:10.967 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.911978913Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T18:36:10.968 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.954023673Z level=info msg="Executing migration" id="create team table" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:10 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:10.979443596Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.004754362Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.038464108Z level=info msg="Executing migration" id="create team member table" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.088301997Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.105396954Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.130722094Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.156051753Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T18:36:11.245 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.231626699Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T18:36:11.514 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[22565]: cluster 2024-09-10T18:36:11.310027+0000 mon.a (mon.0) 816 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:36:11.514 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.257012439Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T18:36:11.515 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.28239878Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T18:36:11.515 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.324475014Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T18:36:11.515 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.366554279Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T18:36:11.515 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.39204678Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T18:36:11.515 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.417279041Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T18:36:11.515 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.459383146Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T18:36:11.515 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.501442819Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T18:36:11.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:11 smithi100 bash[18597]: cluster 2024-09-10T18:36:11.310027+0000 mon.a (mon.0) 816 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:36:11.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:11 smithi100 bash[31607]: cluster 2024-09-10T18:36:11.310027+0000 mon.a (mon.0) 816 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.543502575Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.568940813Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.59064605Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.632692287Z level=info msg="Executing migration" id="create tag table" 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.658051554Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.700110272Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.725430256Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T18:36:11.806 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.79255939Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T18:36:12.065 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:36:12.065 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.814008486Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T18:36:12.065 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.83950438Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T18:36:12.066 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.864810814Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T18:36:12.066 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.909128066Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T18:36:12.066 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.950986579Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T18:36:12.066 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:11 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:11.976305352Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T18:36:12.066 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.006404378Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T18:36:12.066 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.051951522Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T18:36:12.342 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.077338993Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T18:36:12.342 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.102597955Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T18:36:12.343 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.127834734Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T18:36:12.343 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.153160168Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T18:36:12.343 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.17844595Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T18:36:12.343 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.203655646Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T18:36:12.343 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.237197882Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T18:36:12.343 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.262408219Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T18:36:12.343 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.329423691Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T18:36:12.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:12 smithi100 bash[18597]: cluster 2024-09-10T18:36:11.251050+0000 mgr.y (mgr.24512) 63 : cluster [DBG] pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:12.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:12 smithi100 bash[18597]: audit 2024-09-10T18:36:11.924820+0000 mgr.y (mgr.24512) 64 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:12.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:12 smithi100 bash[31607]: cluster 2024-09-10T18:36:11.251050+0000 mgr.y (mgr.24512) 63 : cluster [DBG] pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:12.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:12 smithi100 bash[31607]: audit 2024-09-10T18:36:11.924820+0000 mgr.y (mgr.24512) 64 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:12.619 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[22565]: cluster 2024-09-10T18:36:11.251050+0000 mgr.y (mgr.24512) 63 : cluster [DBG] pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:12.619 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[22565]: audit 2024-09-10T18:36:11.924820+0000 mgr.y (mgr.24512) 64 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:12.619 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.371637542Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T18:36:12.619 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.396880367Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T18:36:12.619 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.437417486Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T18:36:12.620 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.479537562Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T18:36:12.620 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.504822705Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T18:36:12.620 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.53006279Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T18:36:12.620 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.572066227Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T18:36:12.620 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.605814443Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T18:36:12.886 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.647812127Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T18:36:12.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.67303134Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T18:36:12.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.698345507Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T18:36:12.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.723640219Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T18:36:12.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.748874535Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T18:36:12.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.772624523Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T18:36:12.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.797853442Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T18:36:12.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.873240142Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T18:36:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/1137446816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac06f940 msgr2=0x7fd5ac06fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/1137446816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac06f940 0x7fd5ac06fd10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd594004890 tx=0x7fd594009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/1137446816 shutdown_connections 2024-09-10T18:36:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/1137446816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac109400 0x7fd5ac10b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/1137446816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5ac070250 0x7fd5ac108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/1137446816 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac06f940 0x7fd5ac06fd10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/1137446816 >> 172.21.15.100:0/1137446816 conn(0x7fd5ac073590 msgr2=0x7fd5ac073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:13.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/1137446816 shutdown_connections 2024-09-10T18:36:13.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/1137446816 wait complete. 2024-09-10T18:36:13.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 Processor -- start 2024-09-10T18:36:13.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.064+0000 7fd5b1a2c640 1 -- start start 2024-09-10T18:36:13.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac06f940 0x7fd5ac1150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:13.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac070250 0x7fd5ac115600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5ac109400 0x7fd5ac10f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac10f8a0 con 0x7fd5ac06f940 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac10fa10 con 0x7fd5ac109400 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac10fbd0 con 0x7fd5ac070250 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac06f940 0x7fd5ac1150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac06f940 0x7fd5ac1150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52818/0 (socket says 172.21.15.100:52818) 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 -- 172.21.15.100:0/654811069 learned_addr learned my addr 172.21.15.100:0/654811069 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aa7fc640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac070250 0x7fd5ac115600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5ab7fe640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5ac109400 0x7fd5ac10f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 -- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac070250 msgr2=0x7fd5ac115600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac070250 0x7fd5ac115600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 -- 172.21.15.100:0/654811069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5ac109400 msgr2=0x7fd5ac10f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:13.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5ac109400 0x7fd5ac10f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 -- 172.21.15.100:0/654811069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4005040 con 0x7fd5ac06f940 2024-09-10T18:36:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5ab7fe640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5ac109400 0x7fd5ac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T18:36:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5aaffd640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac06f940 0x7fd5ac1150c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5940048f0 tx=0x7fd594009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5a37fe640 1 -- 172.21.15.100:0/654811069 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940092e0 con 0x7fd5ac06f940 2024-09-10T18:36:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5a37fe640 1 -- 172.21.15.100:0/654811069 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd594009570 con 0x7fd5ac06f940 2024-09-10T18:36:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd594005040 con 0x7fd5ac06f940 2024-09-10T18:36:13.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5a37fe640 1 -- 172.21.15.100:0/654811069 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594033690 con 0x7fd5ac06f940 2024-09-10T18:36:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac11bf10 con 0x7fd5ac06f940 2024-09-10T18:36:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.068+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac070e60 con 0x7fd5ac06f940 2024-09-10T18:36:13.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.072+0000 7fd5a37fe640 1 -- 172.21.15.100:0/654811069 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fd594030020 con 0x7fd5ac06f940 2024-09-10T18:36:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.072+0000 7fd5a37fe640 1 --2- 172.21.15.100:0/654811069 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd59806edb0 0x7fd598071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.072+0000 7fd5a37fe640 1 -- 172.21.15.100:0/654811069 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fd5940add90 con 0x7fd5ac06f940 2024-09-10T18:36:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.072+0000 7fd5aa7fc640 1 --2- 172.21.15.100:0/654811069 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd59806edb0 0x7fd598071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:13.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.072+0000 7fd5aa7fc640 1 --2- 172.21.15.100:0/654811069 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd59806edb0 0x7fd598071260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac110980 tx=0x7fd59c00b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:13.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.084+0000 7fd5a37fe640 1 -- 172.21.15.100:0/654811069 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd59407ace0 con 0x7fd5ac06f940 2024-09-10T18:36:13.144 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.898588165Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T18:36:13.144 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.923795827Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T18:36:13.144 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.965836093Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T18:36:13.144 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:12 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:12.990943762Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T18:36:13.144 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.016285671Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T18:36:13.144 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.041491828Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T18:36:13.144 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.083665569Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T18:36:13.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.10892204Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T18:36:13.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.130892389Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T18:36:13.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.264+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5ac04bf60 con 0x7fd5ac06f940 2024-09-10T18:36:13.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.264+0000 7fd5a37fe640 1 -- 172.21.15.100:0/654811069 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+19446 (secure 0 0 0) 0x7fd59407a680 con 0x7fd5ac06f940 2024-09-10T18:36:13.267 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:36:13.267 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":67,"fsid":"505ebbde-6fa1-11ef-bcea-c7b262605968","created":"2024-09-10T18:23:17.238038+0000","modified":"2024-09-10T18:36:10.291516+0000","last_up_change":"2024-09-10T18:33:26.598092+0000","last_in_change":"2024-09-10T18:32:58.920619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:28:49.721821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:34:09.153206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:34:10.904827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:34:12.858132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"datapool","create_time":"2024-09-10T18:34:14.149048+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:34:15.315889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"9a69cb5a-7a42-4fb4-861d-55a459379aa6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6801","nonce":2899184110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2899184110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2899184110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6803","nonce":2899184110}]},"public_addr":"172.21.15.100:6801/2899184110","cluster_addr":"172.21.15.100:6802/2899184110","heartbeat_back_addr":"172.21.15.100:6804/2899184110","heartbeat_front_addr":"172.21.15.100:6803/2899184110","state":["exists","up"]},{"osd":1,"uuid":"b6ad7728-dc92-4291-99cd-18ebfacd6c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6805","nonce":59055423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":59055423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":59055423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6807","nonce":59055423}]},"public_addr":"172.21.15.100:6805/59055423","cluster_addr":"172.21.15.100:6806/59055423","heartbeat_back_addr":"172.21.15.100:6808/59055423","heartbeat_front_addr":"172.21.15.100:6807/59055423","state":["exists","up"]},{"osd":2,"uuid":"ce92944f-0b92-46c8-a6b7-6cdd00f9736a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6809","nonce":370401307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":370401307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":370401307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6811","nonce":370401307}]},"public_addr":"172.21.15.100:6809/370401307","cluster_addr":"172.21.15.100:6810/370401307","heartbeat_back_addr":"172.21.15.100:6812/370401307","heartbeat_front_addr":"172.21.15.100:6811/370401307","state":["exists","up"]},{"osd":3,"uuid":"8340f7e5-8d39-4316-ae1e-809bcf286e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6813","nonce":2754318461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2754318461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":2754318461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6815","nonce":2754318461}]},"public_addr":"172.21.15.100:6813/2754318461","cluster_addr":"172.21.15.100:6814/2754318461","heartbeat_back_addr":"172.21.15.100:6816/2754318461","heartbeat_front_addr":"172.21.15.100:6815/2754318461","state":["exists","up"]},{"osd":4,"uuid":"80851ca1-0c33-4ac6-80b5-92bbcaa3dacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2989412742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6801","nonce":2989412742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6803","nonce":2989412742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2989412742}]},"public_addr":"172.21.15.151:6800/2989412742","cluster_addr":"172.21.15.151:6801/2989412742","heartbeat_back_addr":"172.21.15.151:6803/2989412742","heartbeat_front_addr":"172.21.15.151:6802/2989412742","state":["exists","up"]},{"osd":5,"uuid":"115d5a71-f0c1-4d54-8cb7-7e190a5405a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3330203218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6805","nonce":3330203218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6807","nonce":3330203218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3330203218}]},"public_addr":"172.21.15.151:6804/3330203218","cluster_addr":"172.21.15.151:6805/3330203218","heartbeat_back_addr":"172.21.15.151:6807/3330203218","heartbeat_front_addr":"172.21.15.151:6806/3330203218","state":["exists","up"]},{"osd":6,"uuid":"3dbae406-f3c5-43e0-98bf-9b4d26715d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":251427555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6809","nonce":251427555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6811","nonce":251427555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":251427555}]},"public_addr":"172.21.15.151:6808/251427555","cluster_addr":"172.21.15.151:6809/251427555","heartbeat_back_addr":"172.21.15.151:6811/251427555","heartbeat_front_addr":"172.21.15.151:6810/251427555","state":["exists","up"]},{"osd":7,"uuid":"df3a51c0-0655-462f-b528-e4ccd5d130ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4021034041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6813","nonce":4021034041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6815","nonce":4021034041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4021034041}]},"public_addr":"172.21.15.151:6812/4021034041","cluster_addr":"172.21.15.151:6813/4021034041","heartbeat_back_addr":"172.21.15.151:6815/4021034041","heartbeat_front_addr":"172.21.15.151:6814/4021034041","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:26:57.052592+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:27:51.521280+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:28:46.335623+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:29:42.412430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:30:36.277604+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:31:31.151158+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:32:28.116557+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:33:25.354725+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":7,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/2483032053":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/1686873674":"2024-09-11T18:23:45.175932+0000","172.21.15.100:0/4071543765":"2024-09-11T18:23:45.175932+0000","172.21.15.100:6800/1728522861":"2024-09-11T18:23:45.175932+0000","172.21.15.100:0/1105054798":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/69624316":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/41752804":"2024-09-11T18:23:45.175932+0000","172.21.15.100:6800/2893280940":"2024-09-11T18:24:19.124827+0000","172.21.15.100:6800/3986579393":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/2285226173":"2024-09-11T18:24:19.124827+0000","172.21.15.100:0/124948575":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/1013137826":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/1969858017":"2024-09-11T18:35:09.193020+0000","172.21.15.100:0/2038780913":"2024-09-11T18:35:09.193020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:36:13.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd59806edb0 msgr2=0x7fd598071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:13.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/654811069 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd59806edb0 0x7fd598071260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac110980 tx=0x7fd59c00b040 comp rx=0 tx=0).stop 2024-09-10T18:36:13.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac06f940 msgr2=0x7fd5ac1150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac06f940 0x7fd5ac1150c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5940048f0 tx=0x7fd594009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 shutdown_connections 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/654811069 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd59806edb0 0x7fd598071260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5ac109400 0x7fd5ac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ac070250 0x7fd5ac115600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 --2- 172.21.15.100:0/654811069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ac06f940 0x7fd5ac1150c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 >> 172.21.15.100:0/654811069 conn(0x7fd5ac073590 msgr2=0x7fd5ac0f9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 shutdown_connections 2024-09-10T18:36:13.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:13.268+0000 7fd5b1a2c640 1 -- 172.21.15.100:0/654811069 wait complete. 2024-09-10T18:36:13.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[22565]: audit 2024-09-10T18:36:13.266330+0000 mon.a (mon.0) 817 : audit [DBG] from='client.? 172.21.15.100:0/654811069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:36:13.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.17483435Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T18:36:13.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.200244665Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T18:36:13.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.225469946Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T18:36:13.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.250597868Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T18:36:13.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.275907621Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T18:36:13.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.301119672Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T18:36:13.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.343041133Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T18:36:13.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:13 smithi100 bash[31607]: audit 2024-09-10T18:36:13.266330+0000 mon.a (mon.0) 817 : audit [DBG] from='client.? 172.21.15.100:0/654811069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:36:13.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:13 smithi100 bash[18597]: audit 2024-09-10T18:36:13.266330+0000 mon.a (mon.0) 817 : audit [DBG] from='client.? 172.21.15.100:0/654811069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:36:13.692 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.42688378Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T18:36:13.692 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.485742065Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T18:36:13.693 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.519353588Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T18:36:13.693 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.561298202Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T18:36:13.693 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.586652441Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T18:36:13.693 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.611978953Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T18:36:13.693 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.653814653Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T18:36:13.693 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.679173608Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T18:36:13.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.704479074Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T18:36:13.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.729696118Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T18:36:13.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.755147175Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T18:36:13.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.780445638Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T18:36:13.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.822524667Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T18:36:13.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.847801521Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T18:36:13.991 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.873010325Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T18:36:13.991 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.898318394Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T18:36:13.991 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:13 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:13.976857489Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T18:36:14.242 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.002041007Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T18:36:14.242 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.027329696Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T18:36:14.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.052550124Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T18:36:14.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.103156839Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T18:36:14.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.128471166Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T18:36:14.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.153688332Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T18:36:14.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.178884967Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T18:36:14.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.204223304Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T18:36:14.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.22935858Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T18:36:14.253 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:14 smithi100 systemd[1]: Stopping Ceph alertmanager.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:36:14.261 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T18:36:14.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T18:36:14.262 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T18:36:14.263 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T18:36:14.263 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T18:36:14.263 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T18:36:14.264 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T18:36:14.264 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T18:36:14.504 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:14 smithi100 bash[18597]: cluster 2024-09-10T18:36:13.251740+0000 mgr.y (mgr.24512) 65 : cluster [DBG] pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:36:14.505 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:14 smithi100 bash[31607]: cluster 2024-09-10T18:36:13.251740+0000 mgr.y (mgr.24512) 65 : cluster [DBG] pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:36:14.528 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[22565]: cluster 2024-09-10T18:36:13.251740+0000 mgr.y (mgr.24512) 65 : cluster [DBG] pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:36:14.528 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.262930788Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T18:36:14.528 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.288242242Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T18:36:14.528 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.330042578Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T18:36:14.528 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.355387335Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T18:36:14.528 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.39735245Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T18:36:14.528 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.439391676Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T18:36:14.788 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:36:14.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.514790817Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T18:36:14.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.556767454Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T18:36:14.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.598731025Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T18:36:14.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.640776132Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T18:36:14.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.682684582Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T18:36:14.789 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.707922368Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T18:36:14.789 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.733180916Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T18:36:14.789 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.77515448Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T18:36:14.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:14 smithi100 bash[66166]: ts=2024-09-10T18:36:14.505Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:36:15.061 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.800617392Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T18:36:15.061 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.825847797Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T18:36:15.061 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.851022947Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T18:36:15.061 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.876394156Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T18:36:15.061 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.901605082Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T18:36:15.062 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.943554776Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T18:36:15.062 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.968762418Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T18:36:15.062 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.989268975Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:36:15.062 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:14 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:14.98958357Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:36:15.062 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.04790905Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T18:36:15.313 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.073183022Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T18:36:15.313 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.098459694Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T18:36:15.313 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.098999229Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.099472592Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.123706837Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.157376033Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.182501308Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.207731107Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.249768821Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.274994601Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T18:36:15.314 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.300196595Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T18:36:15.428 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[67844]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:36:15.428 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[68043]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:36:15.428 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:15 smithi100 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@alertmanager.a.service: Succeeded. 2024-09-10T18:36:15.428 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:15 smithi100 systemd[1]: Stopped Ceph alertmanager.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:36:15.428 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:15 smithi100 systemd[1]: Started Ceph alertmanager.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:36:15.599 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.342191482Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T18:36:15.599 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.384278008Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T18:36:15.599 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.426311294Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T18:36:15.599 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.48490805Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T18:36:15.599 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.510208304Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T18:36:15.599 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[22565]: cluster 2024-09-10T18:36:15.376545+0000 mon.a (mon.0) 818 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T18:36:15.599 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[22565]: cluster 2024-09-10T18:36:15.376646+0000 mon.a (mon.0) 819 : cluster [INF] Cluster is now healthy 2024-09-10T18:36:15.599 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[22565]: audit 2024-09-10T18:36:15.404924+0000 mon.a (mon.0) 820 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:15.599 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[22565]: audit 2024-09-10T18:36:15.412384+0000 mon.a (mon.0) 821 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:15.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[31607]: cluster 2024-09-10T18:36:15.376545+0000 mon.a (mon.0) 818 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T18:36:15.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[31607]: cluster 2024-09-10T18:36:15.376646+0000 mon.a (mon.0) 819 : cluster [INF] Cluster is now healthy 2024-09-10T18:36:15.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[31607]: audit 2024-09-10T18:36:15.404924+0000 mon.a (mon.0) 820 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:15.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[31607]: audit 2024-09-10T18:36:15.412384+0000 mon.a (mon.0) 821 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[18597]: cluster 2024-09-10T18:36:15.376545+0000 mon.a (mon.0) 818 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-10T18:36:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[18597]: cluster 2024-09-10T18:36:15.376646+0000 mon.a (mon.0) 819 : cluster [INF] Cluster is now healthy 2024-09-10T18:36:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[18597]: audit 2024-09-10T18:36:15.404924+0000 mon.a (mon.0) 820 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:15 smithi100 bash[18597]: audit 2024-09-10T18:36:15.412384+0000 mon.a (mon.0) 821 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:15.897 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.585430197Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T18:36:15.897 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.644041334Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T18:36:15.897 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.711836233Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T18:36:16.091 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.883961043Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T18:36:16.091 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.957228693Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T18:36:16.091 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:15 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:15.994298737Z level=info msg="Executing migration" id="create permission table" 2024-09-10T18:36:16.091 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:16.077857936Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T18:36:16.574 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:16.244111991Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T18:36:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[22565]: cluster 2024-09-10T18:36:15.253119+0000 mgr.y (mgr.24512) 66 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[22565]: cephadm 2024-09-10T18:36:15.416923+0000 mgr.y (mgr.24512) 67 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:36:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[22565]: cephadm 2024-09-10T18:36:15.426333+0000 mgr.y (mgr.24512) 68 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi151 2024-09-10T18:36:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[18597]: cluster 2024-09-10T18:36:15.253119+0000 mgr.y (mgr.24512) 66 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[18597]: cephadm 2024-09-10T18:36:15.416923+0000 mgr.y (mgr.24512) 67 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:36:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[18597]: cephadm 2024-09-10T18:36:15.426333+0000 mgr.y (mgr.24512) 68 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi151 2024-09-10T18:36:16.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.594Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:36:16.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.594Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:36:16.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.599Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.100 port=9094 2024-09-10T18:36:16.794 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.601Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:36:16.794 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.642Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:36:16.794 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.643Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:36:16.794 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.647Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:36:16.794 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[68068]: ts=2024-09-10T18:36:16.647Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:36:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[31607]: cluster 2024-09-10T18:36:15.253119+0000 mgr.y (mgr.24512) 66 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[31607]: cephadm 2024-09-10T18:36:15.416923+0000 mgr.y (mgr.24512) 67 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:36:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:16 smithi100 bash[31607]: cephadm 2024-09-10T18:36:15.426333+0000 mgr.y (mgr.24512) 68 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi151 2024-09-10T18:36:16.833 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:16.561520091Z level=info msg="Executing migration" id="create role table" 2024-09-10T18:36:16.833 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:16.726983325Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T18:36:17.175 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:16 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:16.819727494Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T18:36:17.175 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.012521145Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T18:36:17.675 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.338111726Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T18:36:17.675 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.396705831Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T18:36:17.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.474952699Z level=info msg="Executing migration" id="create team role table" 2024-09-10T18:36:17.676 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.572765783Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T18:36:17.944 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.673140116Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T18:36:17.944 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.70685158Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T18:36:17.944 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.799062263Z level=info msg="Executing migration" id="create user role table" 2024-09-10T18:36:17.944 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.866119643Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T18:36:17.944 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.908426636Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T18:36:18.243 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:17 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:17.950415616Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T18:36:18.244 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.001107038Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T18:36:18.244 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.062857659Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T18:36:18.244 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.141300384Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T18:36:18.244 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.230485564Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T18:36:18.623 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[22565]: cluster 2024-09-10T18:36:17.253925+0000 mgr.y (mgr.24512) 69 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:18.624 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.397390081Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T18:36:18.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:18 smithi100 bash[18597]: cluster 2024-09-10T18:36:17.253925+0000 mgr.y (mgr.24512) 69 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:18 smithi100 bash[31607]: cluster 2024-09-10T18:36:17.253925+0000 mgr.y (mgr.24512) 69 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:18.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:18 smithi100 bash[68068]: ts=2024-09-10T18:36:18.602Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000453089s 2024-09-10T18:36:18.925 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.610309385Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T18:36:18.925 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.771917399Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T18:36:19.151 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.156 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.157 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.158 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.159 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:19.233 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.916144029Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T18:36:19.233 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:18 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:18.991891817Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T18:36:19.233 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.077738175Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T18:36:19.233 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.101595495Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T18:36:19.233 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.152369797Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T18:36:19.233 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.219374797Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T18:36:19.678 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.406348465Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T18:36:20.175 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.666552244Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T18:36:20.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.796935946Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T18:36:20.176 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:19 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:19.885830856Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T18:36:20.449 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[22565]: cluster 2024-09-10T18:36:19.254660+0000 mgr.y (mgr.24512) 70 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:36:20.449 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.176592472Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T18:36:20.449 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.308415624Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T18:36:20.754 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.436125084Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T18:36:20.754 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.516123191Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T18:36:20.754 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.641076994Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T18:36:20.754 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.740899126Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T18:36:20.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:20 smithi100 bash[18597]: cluster 2024-09-10T18:36:19.254660+0000 mgr.y (mgr.24512) 70 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:36:20.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:20 smithi100 bash[31607]: cluster 2024-09-10T18:36:19.254660+0000 mgr.y (mgr.24512) 70 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:36:21.038 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.788625929Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T18:36:21.038 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.844064989Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T18:36:21.038 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.908752109Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T18:36:21.038 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.95683214Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T18:36:21.038 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:20 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:20.981734461Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T18:36:21.038 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.024989656Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T18:36:21.294 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.099619816Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:36:21.294 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.100084202Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:36:21.294 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.192094493Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:36:21.294 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.19250237Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:36:21.552 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.280904674Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T18:36:21.552 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.372647307Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T18:36:22.146 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:36:22.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.538779736Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:36:22.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:21 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.847826764Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:36:22.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:21.999720908Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:36:22.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.065828678Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:36:22.147 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.133171301Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T18:36:22.399 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 systemd[1]: Stopping Ceph prometheus.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:36:22.399 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.193862118Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T18:36:22.399 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.234415516Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:36:22.399 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.276326766Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:36:22.399 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.326976577Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T18:36:22.662 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[22565]: cluster 2024-09-10T18:36:21.256052+0000 mgr.y (mgr.24512) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 934 B/s rd, 0 op/s 2024-09-10T18:36:22.662 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[22565]: audit 2024-09-10T18:36:21.936069+0000 mgr.y (mgr.24512) 72 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:22.663 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.38559031Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T18:36:22.663 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.461283414Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T18:36:22.663 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.553222894Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T18:36:22.663 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.649080346Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T18:36:22.663 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:36:22.663 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:36:22.663 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.397Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.398Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.399Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.399Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:36:22.664 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[55308]: ts=2024-09-10T18:36:22.399Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:36:22.710 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:22 smithi100 bash[18597]: cluster 2024-09-10T18:36:21.256052+0000 mgr.y (mgr.24512) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 934 B/s rd, 0 op/s 2024-09-10T18:36:22.710 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:22 smithi100 bash[18597]: audit 2024-09-10T18:36:21.936069+0000 mgr.y (mgr.24512) 72 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:22.710 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:22 smithi100 bash[31607]: cluster 2024-09-10T18:36:21.256052+0000 mgr.y (mgr.24512) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 934 B/s rd, 0 op/s 2024-09-10T18:36:22.710 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:22 smithi100 bash[31607]: audit 2024-09-10T18:36:21.936069+0000 mgr.y (mgr.24512) 72 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:22.925 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:22 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:22.8190122Z level=info msg="Executing migration" id="add share column" 2024-09-10T18:36:23.468 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57896]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:36:23.468 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57958]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:36:23.468 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.204436746Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T18:36:23.469 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.317674415Z level=info msg="Executing migration" id="create file table" 2024-09-10T18:36:23.469 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.393484448Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T18:36:23.737 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:23 smithi100 bash[18863]: [10/Sep/2024:18:36:23] ENGINE Bus STOPPING 2024-09-10T18:36:23.737 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:23 smithi100 bash[18863]: [10/Sep/2024:18:36:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:36:23.737 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:23 smithi100 bash[18863]: [10/Sep/2024:18:36:23] ENGINE Bus STOPPED 2024-09-10T18:36:23.737 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:23 smithi100 bash[18863]: [10/Sep/2024:18:36:23] ENGINE Bus STARTING 2024-09-10T18:36:23.737 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:23 smithi100 bash[18863]: [10/Sep/2024:18:36:23] ENGINE Serving on http://:::9283 2024-09-10T18:36:23.737 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:23 smithi100 bash[18863]: [10/Sep/2024:18:36:23] ENGINE Bus STARTED 2024-09-10T18:36:23.792 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:23 smithi100 bash[18863]: [10/Sep/2024:18:36:23] ENGINE Bus STOPPING 2024-09-10T18:36:23.800 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:23 smithi151 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@prometheus.a.service: Succeeded. 2024-09-10T18:36:23.801 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:23 smithi151 systemd[1]: Stopped Ceph prometheus.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:36:23.801 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:23 smithi151 systemd[1]: Started Ceph prometheus.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:36:23.801 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.461108667Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T18:36:23.801 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.5029792Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T18:36:23.801 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.545824116Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T18:36:23.801 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.621425724Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T18:36:24.388 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:23 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:23.787686787Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T18:36:24.388 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:24.084090499Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T18:36:24.388 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:24.30528714Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T18:36:24.388 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:24.374811698Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T18:36:24.513 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:36:24.513 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Bus STOPPED 2024-09-10T18:36:24.513 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Bus STARTING 2024-09-10T18:36:24.513 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Serving on http://:::9283 2024-09-10T18:36:24.513 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Bus STARTED 2024-09-10T18:36:24.513 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Bus STOPPING 2024-09-10T18:36:24.642 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: cluster 2024-09-10T18:36:23.256745+0000 mgr.y (mgr.24512) 73 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.516659+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.526413+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.533347+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.534236+0000 mgr.y (mgr.24512) 74 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.536366+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.537175+0000 mgr.y (mgr.24512) 75 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.546639+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.564672+0000 mon.c (mon.2) 66 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.565183+0000 mgr.y (mgr.24512) 76 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.567408+0000 mon.c (mon.2) 67 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.567897+0000 mgr.y (mgr.24512) 77 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.573858+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.643 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.584669+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:36:24.644 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.585317+0000 mgr.y (mgr.24512) 78 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:36:24.644 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.588623+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:36:24.644 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.589226+0000 mgr.y (mgr.24512) 79 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:36:24.644 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.596982+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: cluster 2024-09-10T18:36:23.256745+0000 mgr.y (mgr.24512) 73 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:24.778 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.516659+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.526413+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.533347+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.534236+0000 mgr.y (mgr.24512) 74 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.536366+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.537175+0000 mgr.y (mgr.24512) 75 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.546639+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.564672+0000 mon.c (mon.2) 66 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.565183+0000 mgr.y (mgr.24512) 76 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:36:24.779 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.567408+0000 mon.c (mon.2) 67 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.567897+0000 mgr.y (mgr.24512) 77 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.573858+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.584669+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.585317+0000 mgr.y (mgr.24512) 78 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.588623+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.589226+0000 mgr.y (mgr.24512) 79 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.596982+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:23.605003+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:24.780 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18597]: audit 2024-09-10T18:36:24.315140+0000 mon.c (mon.2) 71 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: cluster 2024-09-10T18:36:23.256745+0000 mgr.y (mgr.24512) 73 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.516659+0000 mon.a (mon.0) 822 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.526413+0000 mon.a (mon.0) 823 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.533347+0000 mon.c (mon.2) 64 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.534236+0000 mgr.y (mgr.24512) 74 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.536366+0000 mon.c (mon.2) 65 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.537175+0000 mgr.y (mgr.24512) 75 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.546639+0000 mon.a (mon.0) 824 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.564672+0000 mon.c (mon.2) 66 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.565183+0000 mgr.y (mgr.24512) 76 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.567408+0000 mon.c (mon.2) 67 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.567897+0000 mgr.y (mgr.24512) 77 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.573858+0000 mon.a (mon.0) 825 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.781 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.584669+0000 mon.c (mon.2) 68 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:36:24.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.585317+0000 mgr.y (mgr.24512) 78 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:36:24.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.588623+0000 mon.c (mon.2) 69 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:36:24.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.589226+0000 mgr.y (mgr.24512) 79 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:36:24.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.596982+0000 mon.a (mon.0) 826 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:24.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:23.605003+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:24.782 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[31607]: audit 2024-09-10T18:36:24.315140+0000 mon.c (mon.2) 71 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:23.605003+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:24.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[22565]: audit 2024-09-10T18:36:24.315140+0000 mon.c (mon.2) 71 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:24.926 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:24.62850025Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T18:36:25.043 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:36:25.043 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Bus STOPPED 2024-09-10T18:36:25.043 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Bus STARTING 2024-09-10T18:36:25.043 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Serving on http://:::9283 2024-09-10T18:36:25.043 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:24 smithi100 bash[18863]: [10/Sep/2024:18:36:24] ENGINE Bus STARTED 2024-09-10T18:36:25.228 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:24 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:24.933751212Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T18:36:25.229 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.014469323Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T18:36:25.229 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.098630374Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T18:36:25.229 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.215515909Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T18:36:25.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.332701844Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T18:36:25.530 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.41678575Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.517156013Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.567908092Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.647256308Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.710806615Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.736115626Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.769836715Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.667Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:36:25.783 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.667Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.667Z caller=main.go:570 level=info host_details="(Linux 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 smithi151 (none))" 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.667Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.667Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.671Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.672Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.674Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.675Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.678Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.678Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.372µs 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.678Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.679Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.680Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T18:36:25.784 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.680Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=81.688µs wal_replay_duration=1.246588ms wbl_replay_duration=133ns total_replay_duration=1.384944ms 2024-09-10T18:36:25.785 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.681Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:36:25.785 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.681Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:36:25.785 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.681Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:36:25.785 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.708Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=27.179148ms db_storage=1.107µs remote_storage=1.659µs web_handler=483ns query_engine=1.142µs scrape=370.07µs scrape_sd=43.693µs notify=30.845µs notify_sd=15.45µs rules=26.275588ms tracing=12.998µs 2024-09-10T18:36:25.785 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.708Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:36:25.785 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57984]: ts=2024-09-10T18:36:25.708Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:36:26.035 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.794923165Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T18:36:26.035 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.820285543Z level=info msg="Executing migration" id="create folder table" 2024-09-10T18:36:26.035 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.862173711Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T18:36:26.035 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.921688922Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T18:36:26.035 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:25 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:25.9468675Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T18:36:26.036 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:26.022170021Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T18:36:26.425 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=migrator t=2024-09-10T18:36:26.056081885Z level=info msg="migrations completed" performed=485 skipped=0 duration=23.79261887s 2024-09-10T18:36:26.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=sqlstore t=2024-09-10T18:36:26.057558657Z level=info msg="Created default organization" 2024-09-10T18:36:26.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=plugin.loader t=2024-09-10T18:36:26.165510108Z level=info msg="Plugin registered" pluginID=input 2024-09-10T18:36:26.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=plugin.loader t=2024-09-10T18:36:26.194447719Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T18:36:26.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=plugin.loader t=2024-09-10T18:36:26.194474474Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T18:36:26.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=secrets t=2024-09-10T18:36:26.194513777Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T18:36:26.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=query_data t=2024-09-10T18:36:26.196408011Z level=info msg="Query Service initialization" 2024-09-10T18:36:26.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[57256]: logger=live.push_http t=2024-09-10T18:36:26.200619599Z level=info msg="Live Push Gateway initialization" 2024-09-10T18:36:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:26 smithi100 bash[18597]: cluster 2024-09-10T18:36:25.258228+0000 mgr.y (mgr.24512) 80 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:26.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:26 smithi100 bash[31607]: cluster 2024-09-10T18:36:25.258228+0000 mgr.y (mgr.24512) 80 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:26.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:36:26 smithi100 bash[68068]: ts=2024-09-10T18:36:26.604Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002899789s 2024-09-10T18:36:26.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:26 smithi151 bash[22565]: cluster 2024-09-10T18:36:25.258228+0000 mgr.y (mgr.24512) 80 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=infra.usagestats.collector t=2024-09-10T18:36:27.134007796Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=provisioning.datasources t=2024-09-10T18:36:27.134452955Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=provisioning.datasources t=2024-09-10T18:36:27.168544675Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=provisioning.datasources t=2024-09-10T18:36:27.231019146Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=provisioning.alerting t=2024-09-10T18:36:27.296603045Z level=info msg="starting to provision alerting" 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=provisioning.alerting t=2024-09-10T18:36:27.296670621Z level=info msg="finished to provision alerting" 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=grafanaStorageLogger t=2024-09-10T18:36:27.29716458Z level=info msg="storage starting" 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=ngalert.state.manager t=2024-09-10T18:36:27.297592033Z level=info msg="Warming state cache for startup" 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=http.server t=2024-09-10T18:36:27.302621734Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=sqlstore.transactions t=2024-09-10T18:36:27.308013966Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:36:27.426 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=ngalert.state.manager t=2024-09-10T18:36:27.341658572Z level=info msg="State cache has been initialized" states=0 duration=44.065715ms 2024-09-10T18:36:27.427 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=ngalert.multiorg.alertmanager t=2024-09-10T18:36:27.341761833Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T18:36:27.427 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:36:27 smithi151 bash[57256]: logger=ticker t=2024-09-10T18:36:27.341830652Z level=info msg=starting first_tick=2024-09-10T18:36:30Z 2024-09-10T18:36:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:28 smithi100 bash[18597]: cluster 2024-09-10T18:36:27.258982+0000 mgr.y (mgr.24512) 81 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:28 smithi100 bash[31607]: cluster 2024-09-10T18:36:27.258982+0000 mgr.y (mgr.24512) 81 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:28.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:28 smithi151 bash[22565]: cluster 2024-09-10T18:36:27.258982+0000 mgr.y (mgr.24512) 81 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:29.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- 172.21.15.100:0/2839965444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50cc100ee0 msgr2=0x7f50cc1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:29.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 --2- 172.21.15.100:0/2839965444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50cc100ee0 0x7f50cc1012b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f50bc004950 tx=0x7f50bc009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:29.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- 172.21.15.100:0/2839965444 shutdown_connections 2024-09-10T18:36:29.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 --2- 172.21.15.100:0/2839965444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50cc0fb790 0x7f50cc107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:29.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 --2- 172.21.15.100:0/2839965444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50cc101880 0x7f50cc0fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:29.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 --2- 172.21.15.100:0/2839965444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50cc100ee0 0x7f50cc1012b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:29.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- 172.21.15.100:0/2839965444 >> 172.21.15.100:0/2839965444 conn(0x7f50cc071c60 msgr2=0x7f50cc072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:29.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- 172.21.15.100:0/2839965444 shutdown_connections 2024-09-10T18:36:29.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- 172.21.15.100:0/2839965444 wait complete. 2024-09-10T18:36:29.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 Processor -- start 2024-09-10T18:36:29.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- start start 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50cc0fb790 0x7f50cc112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50cc100ee0 0x7f50cc113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50cc101880 0x7f50cc10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc10d5f0 con 0x7f50cc100ee0 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc10d760 con 0x7f50cc101880 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50d0a98640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc10d920 con 0x7f50cc0fb790 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50cc0fb790 0x7f50cc112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50cc0fb790 0x7f50cc112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:42824/0 (socket says 172.21.15.100:42824) 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 -- 172.21.15.100:0/1627590933 learned_addr learned my addr 172.21.15.100:0/1627590933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50c9d9b640 1 --2- 172.21.15.100:0/1627590933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50cc100ee0 0x7f50cc113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:29.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50cad9d640 1 --2- 172.21.15.100:0/1627590933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50cc101880 0x7f50cc10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:29.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 -- 172.21.15.100:0/1627590933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50cc101880 msgr2=0x7f50cc10cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:29.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 --2- 172.21.15.100:0/1627590933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50cc101880 0x7f50cc10cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:29.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 -- 172.21.15.100:0/1627590933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50cc100ee0 msgr2=0x7f50cc113350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:29.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 --2- 172.21.15.100:0/1627590933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50cc100ee0 0x7f50cc113350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:29.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 -- 172.21.15.100:0/1627590933 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc005040 con 0x7f50cc0fb790 2024-09-10T18:36:29.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50ca59c640 1 --2- 172.21.15.100:0/1627590933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50cc0fb790 0x7f50cc112e10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0043d0 tx=0x7f50bc035040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:29.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.376+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc009900 con 0x7f50cc0fb790 2024-09-10T18:36:29.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f50bc02ab70 con 0x7f50cc0fb790 2024-09-10T18:36:29.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc10dba0 con 0x7f50cc0fb790 2024-09-10T18:36:29.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc033420 con 0x7f50cc0fb790 2024-09-10T18:36:29.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc119e00 con 0x7f50cc0fb790 2024-09-10T18:36:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f50bc030020 con 0x7f50cc0fb790 2024-09-10T18:36:29.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50cc100430 con 0x7f50cc0fb790 2024-09-10T18:36:29.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.100:6800/2155730476 conn(0x7f50b406edb0 0x7f50b4071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:29.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f50bc0ae1c0 con 0x7f50cc0fb790 2024-09-10T18:36:29.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50c9d9b640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.100:6800/2155730476 conn(0x7f50b406edb0 0x7f50b4071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:29.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.151:6812/4021034041 conn(0x7f50b4078630 0x7f50b407aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:29.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 --> v2:172.21.15.151:6812/4021034041 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50b407b150 con 0x7f50b4078630 2024-09-10T18:36:29.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.380+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50bc0b0850 con 0x7f50cc0fb790 2024-09-10T18:36:29.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.384+0000 7f50cad9d640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.151:6812/4021034041 conn(0x7f50b4078630 0x7f50b407aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:29.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.384+0000 7f50c9d9b640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.100:6800/2155730476 conn(0x7f50b406edb0 0x7f50b4071260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f50cc10e6d0 tx=0x7f50c0009040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:29.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.384+0000 7f50cad9d640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.151:6812/4021034041 conn(0x7f50b4078630 0x7f50b407aac0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:29.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.384+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== osd.7 v2:172.21.15.151:6812/4021034041 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f50b407b150 con 0x7f50b4078630 2024-09-10T18:36:29.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.400+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 --> v2:172.21.15.151:6812/4021034041 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f50cc0fc660 con 0x7f50b4078630 2024-09-10T18:36:29.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50baffd640 1 -- 172.21.15.100:0/1627590933 <== osd.7 v2:172.21.15.151:6812/4021034041 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f50cc0fc660 con 0x7f50b4078630 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 >> v2:172.21.15.151:6812/4021034041 conn(0x7f50b4078630 msgr2=0x7f50b407aac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.151:6812/4021034041 conn(0x7f50b4078630 0x7f50b407aac0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 >> v2:172.21.15.100:6800/2155730476 conn(0x7f50b406edb0 msgr2=0x7f50b4071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 --2- 172.21.15.100:0/1627590933 >> v2:172.21.15.100:6800/2155730476 conn(0x7f50b406edb0 0x7f50b4071260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f50cc10e6d0 tx=0x7f50c0009040 comp rx=0 tx=0).stop 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50cc0fb790 msgr2=0x7f50cc112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 --2- 172.21.15.100:0/1627590933 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50cc0fb790 0x7f50cc112e10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0043d0 tx=0x7f50bc035040 comp rx=0 tx=0).stop 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50cad9d640 1 -- 172.21.15.100:0/1627590933 reap_dead start 2024-09-10T18:36:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 shutdown_connections 2024-09-10T18:36:29.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 >> 172.21.15.100:0/1627590933 conn(0x7f50cc071c60 msgr2=0x7f50cc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:29.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 shutdown_connections 2024-09-10T18:36:29.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:29.404+0000 7f50d0a98640 1 -- 172.21.15.100:0/1627590933 wait complete. 2024-09-10T18:36:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:30 smithi100 bash[18597]: cluster 2024-09-10T18:36:29.259621+0000 mgr.y (mgr.24512) 82 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:30.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:30 smithi100 bash[31607]: cluster 2024-09-10T18:36:29.259621+0000 mgr.y (mgr.24512) 82 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:30.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:30 smithi151 bash[22565]: cluster 2024-09-10T18:36:29.259621+0000 mgr.y (mgr.24512) 82 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:31] "GET /metrics HTTP/1.1" 200 219714 "" "Prometheus/2.43.0" 2024-09-10T18:36:32.163 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:31 smithi100 bash[18597]: audit 2024-09-10T18:36:30.845751+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:32.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:31 smithi100 bash[18597]: audit 2024-09-10T18:36:30.855958+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:32.164 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:31 smithi100 bash[31607]: audit 2024-09-10T18:36:30.845751+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:32.165 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:31 smithi100 bash[31607]: audit 2024-09-10T18:36:30.855958+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:32.166 INFO:teuthology.orchestra.run.smithi100.stdout:210453397542 2024-09-10T18:36:32.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T18:36:32.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:36:31 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:36:32.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:31 smithi151 bash[22565]: audit 2024-09-10T18:36:30.845751+0000 mon.a (mon.0) 827 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:32.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:31 smithi151 bash[22565]: audit 2024-09-10T18:36:30.855958+0000 mon.a (mon.0) 828 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:32.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- 172.21.15.100:0/3985298390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f100652c0 msgr2=0x7f9f10065710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 --2- 172.21.15.100:0/3985298390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f100652c0 0x7f9f10065710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8004950 tx=0x7f9ef8009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- 172.21.15.100:0/3985298390 shutdown_connections 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 --2- 172.21.15.100:0/3985298390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10065c50 0x7f9f101095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 --2- 172.21.15.100:0/3985298390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f100652c0 0x7f9f10065710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 --2- 172.21.15.100:0/3985298390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f10102870 0x7f9f10102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- 172.21.15.100:0/3985298390 >> 172.21.15.100:0/3985298390 conn(0x7f9f10072440 msgr2=0x7f9f10072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- 172.21.15.100:0/3985298390 shutdown_connections 2024-09-10T18:36:32.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- 172.21.15.100:0/3985298390 wait complete. 2024-09-10T18:36:32.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 Processor -- start 2024-09-10T18:36:32.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- start start 2024-09-10T18:36:32.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f10065c50 0x7f9f10112ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f10102870 0x7f9f101133e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f1010d030 0x7f9f1010d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1010da10 con 0x7f9f10102870 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1010db80 con 0x7f9f1010d030 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f15520640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1010dd40 con 0x7f9f10065c50 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f10065c50 0x7f9f10112ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f10065c50 0x7f9f10112ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41276/0 (socket says 172.21.15.100:41276) 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0effd640 1 -- 172.21.15.100:0/3715275426 learned_addr learned my addr 172.21.15.100:0/3715275426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0e7fc640 1 --2- 172.21.15.100:0/3715275426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f10102870 0x7f9f101133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0f7fe640 1 --2- 172.21.15.100:0/3715275426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f1010d030 0x7f9f1010d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0e7fc640 1 -- 172.21.15.100:0/3715275426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f10065c50 msgr2=0x7f9f10112ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0e7fc640 1 --2- 172.21.15.100:0/3715275426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f10065c50 0x7f9f10112ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0e7fc640 1 -- 172.21.15.100:0/3715275426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f1010d030 msgr2=0x7f9f1010d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.344+0000 7f9f0e7fc640 1 --2- 172.21.15.100:0/3715275426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f1010d030 0x7f9f1010d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f0e7fc640 1 -- 172.21.15.100:0/3715275426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8005040 con 0x7f9f10102870 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f0e7fc640 1 --2- 172.21.15.100:0/3715275426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f10102870 0x7f9f101133e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8008000 tx=0x7f9ef8039040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef802a960 con 0x7f9f10102870 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f100fb950 con 0x7f9f10102870 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f9ef8037830 con 0x7f9f10102870 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8032830 con 0x7f9f10102870 2024-09-10T18:36:32.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f100fbde0 con 0x7f9f10102870 2024-09-10T18:36:32.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f9ef8030020 con 0x7f9f10102870 2024-09-10T18:36:32.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9f10101d00 con 0x7f9f10102870 2024-09-10T18:36:32.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f077fe640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9efc06edb0 0x7f9efc071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f89cc56e640 1 -- 172.21.15.100:0/3259340207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 msgr2=0x7f89c41085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f89cc56e640 1 --2- 172.21.15.100:0/3259340207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 0x7f89c41085a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009cd0 tx=0x7f89bc004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:32.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f9ef80b1ee0 con 0x7f9f10102870 2024-09-10T18:36:32.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f89cc56e640 1 -- 172.21.15.100:0/3259340207 shutdown_connections 2024-09-10T18:36:32.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f89cc56e640 1 --2- 172.21.15.100:0/3259340207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 0x7f89c41085a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f9f0effd640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9efc06edb0 0x7f9efc071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f9f077fe640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.151:6800/2989412742 conn(0x7f9efc078530 0x7f9efc07a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 --> v2:172.21.15.151:6800/2989412742 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9efc07b050 con 0x7f9efc078530 2024-09-10T18:36:32.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f89cc56e640 1 --2- 172.21.15.100:0/3259340207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89c4102120 0x7f89c40fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f89cc56e640 1 --2- 172.21.15.100:0/3259340207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4101780 0x7f89c4101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.348+0000 7f89cc56e640 1 -- 172.21.15.100:0/3259340207 >> 172.21.15.100:0/3259340207 conn(0x7f89c4072440 msgr2=0x7f89c4072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:32.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef80b2200 con 0x7f9f10102870 2024-09-10T18:36:32.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- 172.21.15.100:0/3259340207 shutdown_connections 2024-09-10T18:36:32.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- 172.21.15.100:0/3259340207 wait complete. 2024-09-10T18:36:32.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f9f0f7fe640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.151:6800/2989412742 conn(0x7f9efc078530 0x7f9efc07a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 Processor -- start 2024-09-10T18:36:32.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f9f0effd640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9efc06edb0 0x7f9efc071260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9f00004fa0 tx=0x7f9f0000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- start start 2024-09-10T18:36:32.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 0x7f89c4110940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89c4101780 0x7f89c4110e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4102120 0x7f89c410aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c410b140 con 0x7f89c4102120 2024-09-10T18:36:32.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c410b280 con 0x7f89c4101780 2024-09-10T18:36:32.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c410b440 con 0x7f89c40fc110 2024-09-10T18:36:32.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89ca30a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 0x7f89c4110940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cab0b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4102120 0x7f89c410aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f9f0f7fe640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.151:6800/2989412742 conn(0x7f9efc078530 0x7f9efc07a9c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89ca30a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 0x7f89c4110940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41306/0 (socket says 172.21.15.100:41306) 2024-09-10T18:36:32.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89ca30a640 1 -- 172.21.15.100:0/4121840300 learned_addr learned my addr 172.21.15.100:0/4121840300 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89c9b09640 1 --2- 172.21.15.100:0/4121840300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89c4101780 0x7f89c4110e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cab0b640 1 -- 172.21.15.100:0/4121840300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 msgr2=0x7f89c4110940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cab0b640 1 --2- 172.21.15.100:0/4121840300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 0x7f89c4110940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cab0b640 1 -- 172.21.15.100:0/4121840300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89c4101780 msgr2=0x7f89c4110e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cab0b640 1 --2- 172.21.15.100:0/4121840300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89c4101780 0x7f89c4110e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cab0b640 1 -- 172.21.15.100:0/4121840300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b4005040 con 0x7f89c4102120 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89c9b09640 1 --2- 172.21.15.100:0/4121840300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89c4101780 0x7f89c4110e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89ca30a640 1 --2- 172.21.15.100:0/4121840300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89c40fc110 0x7f89c4110940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cab0b640 1 --2- 172.21.15.100:0/4121840300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4102120 0x7f89c410aaf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89bc02a8d0 tx=0x7f89bc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc038070 con 0x7f89c4102120 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc0098b0 con 0x7f89c4102120 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f89bc033e30 con 0x7f89c4102120 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc03db10 con 0x7f89c4102120 2024-09-10T18:36:32.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c4067130 con 0x7f89c4102120 2024-09-10T18:36:32.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89c40fb2d0 con 0x7f89c4102120 2024-09-10T18:36:32.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.352+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== osd.4 v2:172.21.15.151:6800/2989412742 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9efc07b050 con 0x7f9efc078530 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f89bc030070 con 0x7f89c4102120 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89baffd640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.100:6800/2155730476 conn(0x7f89b006eec0 0x7f89b0071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89ca30a640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.100:6800/2155730476 conn(0x7f89b006eec0 0x7f89b0071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f89bc0aed40 con 0x7f89c4102120 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89baffd640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.151:6808/251427555 conn(0x7f89b0078830 0x7f89b007acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 --> v2:172.21.15.151:6808/251427555 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89b007b350 con 0x7f89b0078830 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89bc0b2020 con 0x7f89c4102120 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89ca30a640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.100:6800/2155730476 conn(0x7f89b006eec0 0x7f89b0071370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f89b4010b90 tx=0x7f89b400d040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89c9b09640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.151:6808/251427555 conn(0x7f89b0078830 0x7f89b007acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.356+0000 7f89c9b09640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.151:6808/251427555 conn(0x7f89b0078830 0x7f89b007acc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.360+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== osd.6 v2:172.21.15.151:6808/251427555 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f89b007b350 con 0x7f89b0078830 2024-09-10T18:36:32.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 --> v2:172.21.15.151:6800/2989412742 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9f101020b0 con 0x7f9efc078530 2024-09-10T18:36:32.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f077fe640 1 -- 172.21.15.100:0/3715275426 <== osd.4 v2:172.21.15.151:6800/2989412742 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9f101020b0 con 0x7f9efc078530 2024-09-10T18:36:32.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 >> v2:172.21.15.151:6800/2989412742 conn(0x7f9efc078530 msgr2=0x7f9efc07a9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.151:6800/2989412742 conn(0x7f9efc078530 0x7f9efc07a9c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9efc06edb0 msgr2=0x7f9efc071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 --2- 172.21.15.100:0/3715275426 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9efc06edb0 0x7f9efc071260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9f00004fa0 tx=0x7f9f0000b040 comp rx=0 tx=0).stop 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f10102870 msgr2=0x7f9f101133e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 --2- 172.21.15.100:0/3715275426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f10102870 0x7f9f101133e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8008000 tx=0x7f9ef8039040 comp rx=0 tx=0).stop 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f0f7fe640 1 -- 172.21.15.100:0/3715275426 reap_dead start 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 shutdown_connections 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 >> 172.21.15.100:0/3715275426 conn(0x7f9f10072440 msgr2=0x7f9f10100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:32.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 shutdown_connections 2024-09-10T18:36:32.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.372+0000 7f9f15520640 1 -- 172.21.15.100:0/3715275426 wait complete. 2024-09-10T18:36:32.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 --> v2:172.21.15.151:6808/251427555 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89c40fced0 con 0x7f89b0078830 2024-09-10T18:36:32.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89baffd640 1 -- 172.21.15.100:0/4121840300 <== osd.6 v2:172.21.15.151:6808/251427555 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f89c40fced0 con 0x7f89b0078830 2024-09-10T18:36:32.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 >> v2:172.21.15.151:6808/251427555 conn(0x7f89b0078830 msgr2=0x7f89b007acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cc56e640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.151:6808/251427555 conn(0x7f89b0078830 0x7f89b007acc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 >> v2:172.21.15.100:6800/2155730476 conn(0x7f89b006eec0 msgr2=0x7f89b0071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cc56e640 1 --2- 172.21.15.100:0/4121840300 >> v2:172.21.15.100:6800/2155730476 conn(0x7f89b006eec0 0x7f89b0071370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f89b4010b90 tx=0x7f89b400d040 comp rx=0 tx=0).stop 2024-09-10T18:36:32.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4102120 msgr2=0x7f89c410aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cc56e640 1 --2- 172.21.15.100:0/4121840300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4102120 0x7f89c410aaf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89bc02a8d0 tx=0x7f89bc004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:32.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.376+0000 7f89cab0b640 1 -- 172.21.15.100:0/4121840300 reap_dead start 2024-09-10T18:36:32.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.380+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 shutdown_connections 2024-09-10T18:36:32.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.380+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 >> 172.21.15.100:0/4121840300 conn(0x7f89c4072440 msgr2=0x7f89c40fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:32.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.380+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 shutdown_connections 2024-09-10T18:36:32.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.380+0000 7f89cc56e640 1 -- 172.21.15.100:0/4121840300 wait complete. 2024-09-10T18:36:32.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- 172.21.15.100:0/1787589454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee981090a0 msgr2=0x7fee9810b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 --2- 172.21.15.100:0/1787589454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee981090a0 0x7fee9810b480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee90009cd0 tx=0x7fee90004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:32.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- 172.21.15.100:0/1787589454 shutdown_connections 2024-09-10T18:36:32.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 --2- 172.21.15.100:0/1787589454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee981090a0 0x7fee9810b480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 --2- 172.21.15.100:0/1787589454 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee98066490 0x7fee98066900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 --2- 172.21.15.100:0/1787589454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee980fd770 0x7fee98065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- 172.21.15.100:0/1787589454 >> 172.21.15.100:0/1787589454 conn(0x7fee98072430 msgr2=0x7fee98072830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:32.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- 172.21.15.100:0/1787589454 shutdown_connections 2024-09-10T18:36:32.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- 172.21.15.100:0/1787589454 wait complete. 2024-09-10T18:36:32.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 Processor -- start 2024-09-10T18:36:32.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- start start 2024-09-10T18:36:32.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee98066490 0x7fee98114dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee980fd770 0x7fee98115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee981090a0 0x7fee9810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9810f5a0 con 0x7fee98066490 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9810f710 con 0x7fee980fd770 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee9d806640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9810f8d0 con 0x7fee981090a0 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee96ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee98066490 0x7fee98114dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee96ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee98066490 0x7fee98114dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38244/0 (socket says 172.21.15.100:38244) 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee96ffd640 1 -- 172.21.15.100:0/1115068734 learned_addr learned my addr 172.21.15.100:0/1115068734 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.776+0000 7fee977fe640 1 --2- 172.21.15.100:0/1115068734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee981090a0 0x7fee9810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee96ffd640 1 -- 172.21.15.100:0/1115068734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee981090a0 msgr2=0x7fee9810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee96ffd640 1 --2- 172.21.15.100:0/1115068734 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee981090a0 0x7fee9810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee96ffd640 1 -- 172.21.15.100:0/1115068734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee980fd770 msgr2=0x7fee98115300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee96ffd640 1 --2- 172.21.15.100:0/1115068734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee980fd770 0x7fee98115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee96ffd640 1 -- 172.21.15.100:0/1115068734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee900098b0 con 0x7fee98066490 2024-09-10T18:36:32.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee96ffd640 1 --2- 172.21.15.100:0/1115068734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee98066490 0x7fee98114dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fee88002ad0 tx=0x7fee88004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88007c30 con 0x7fee98066490 2024-09-10T18:36:32.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9810fbb0 con 0x7fee98066490 2024-09-10T18:36:32.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fee8800f040 con 0x7fee98066490 2024-09-10T18:36:32.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9811bdb0 con 0x7fee98066490 2024-09-10T18:36:32.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88009320 con 0x7fee98066490 2024-09-10T18:36:32.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.780+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee84000f20 con 0x7fee98066490 2024-09-10T18:36:32.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fee880096d0 con 0x7fee98066490 2024-09-10T18:36:32.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee7f7fe640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6800/2155730476 conn(0x7fee8006eec0 0x7fee80071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fee8808ede0 con 0x7fee98066490 2024-09-10T18:36:32.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee7f7fe640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6805/59055423 conn(0x7fee80078830 0x7fee8007acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:32.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 --> v2:172.21.15.100:6805/59055423 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee8007b350 con 0x7fee80078830 2024-09-10T18:36:32.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee967fc640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6800/2155730476 conn(0x7fee8006eec0 0x7fee80071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8808f0d0 con 0x7fee98066490 2024-09-10T18:36:32.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee977fe640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6805/59055423 conn(0x7fee80078830 0x7fee8007acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:32.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee977fe640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6805/59055423 conn(0x7fee80078830 0x7fee8007acc0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.784+0000 7fee967fc640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6800/2155730476 conn(0x7fee8006eec0 0x7fee80071370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee98110680 tx=0x7fee8c00b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:32.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.788+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== osd.1 v2:172.21.15.100:6805/59055423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fee8007b350 con 0x7fee80078830 2024-09-10T18:36:32.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 --> v2:172.21.15.100:6805/59055423 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee84002b40 con 0x7fee80078830 2024-09-10T18:36:32.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee7f7fe640 1 -- 172.21.15.100:0/1115068734 <== osd.1 v2:172.21.15.100:6805/59055423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee84002b40 con 0x7fee80078830 2024-09-10T18:36:32.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6805/59055423 conn(0x7fee80078830 msgr2=0x7fee8007acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6805/59055423 conn(0x7fee80078830 0x7fee8007acc0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6800/2155730476 conn(0x7fee8006eec0 msgr2=0x7fee80071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 --2- 172.21.15.100:0/1115068734 >> v2:172.21.15.100:6800/2155730476 conn(0x7fee8006eec0 0x7fee80071370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee98110680 tx=0x7fee8c00b040 comp rx=0 tx=0).stop 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee98066490 msgr2=0x7fee98114dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 --2- 172.21.15.100:0/1115068734 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee98066490 0x7fee98114dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fee88002ad0 tx=0x7fee88004600 comp rx=0 tx=0).stop 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee977fe640 1 -- 172.21.15.100:0/1115068734 reap_dead start 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 shutdown_connections 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 >> 172.21.15.100:0/1115068734 conn(0x7fee98072430 msgr2=0x7fee980faa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 shutdown_connections 2024-09-10T18:36:32.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:32.804+0000 7fee9d806640 1 -- 172.21.15.100:0/1115068734 wait complete. 2024-09-10T18:36:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:32 smithi100 bash[18597]: cluster 2024-09-10T18:36:31.261053+0000 mgr.y (mgr.24512) 83 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:32 smithi100 bash[18597]: audit 2024-09-10T18:36:31.947462+0000 mgr.y (mgr.24512) 84 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:32 smithi100 bash[31607]: cluster 2024-09-10T18:36:31.261053+0000 mgr.y (mgr.24512) 83 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:32 smithi100 bash[31607]: audit 2024-09-10T18:36:31.947462+0000 mgr.y (mgr.24512) 84 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:33.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:32 smithi151 bash[22565]: cluster 2024-09-10T18:36:31.261053+0000 mgr.y (mgr.24512) 83 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:33.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:32 smithi151 bash[22565]: audit 2024-09-10T18:36:31.947462+0000 mgr.y (mgr.24512) 84 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:33.956 INFO:teuthology.orchestra.run.smithi100.stdout:133143986249 2024-09-10T18:36:33.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T18:36:34.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 -- 172.21.15.100:0/1578382606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fc9e0 msgr2=0x7fef600fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:34.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 --2- 172.21.15.100:0/1578382606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fc9e0 0x7fef600fce50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef48009d00 tx=0x7fef48004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:34.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 -- 172.21.15.100:0/1578382606 shutdown_connections 2024-09-10T18:36:34.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 --2- 172.21.15.100:0/1578382606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef600fd390 0x7fef601095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:34.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 --2- 172.21.15.100:0/1578382606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fc9e0 0x7fef600fce50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:34.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 --2- 172.21.15.100:0/1578382606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef60102870 0x7fef60102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:34.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 -- 172.21.15.100:0/1578382606 >> 172.21.15.100:0/1578382606 conn(0x7fef600747e0 msgr2=0x7fef60074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:34.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 -- 172.21.15.100:0/1578382606 shutdown_connections 2024-09-10T18:36:34.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 -- 172.21.15.100:0/1578382606 wait complete. 2024-09-10T18:36:34.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 Processor -- start 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.036+0000 7fef65c37640 1 -- start start 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef600fc9e0 0x7fef60112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fd390 0x7fef60113310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef60102870 0x7fef6010cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6010d5b0 con 0x7fef600fd390 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6010d720 con 0x7fef600fc9e0 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6010d8e0 con 0x7fef60102870 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef60102870 0x7fef6010cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fd390 0x7fef60113310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fd390 0x7fef60113310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38266/0 (socket says 172.21.15.100:38266) 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef60102870 0x7fef6010cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41326/0 (socket says 172.21.15.100:41326) 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 -- 172.21.15.100:0/2094208181 learned_addr learned my addr 172.21.15.100:0/2094208181 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5f7fe640 1 --2- 172.21.15.100:0/2094208181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef600fc9e0 0x7fef60112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 -- 172.21.15.100:0/2094208181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef60102870 msgr2=0x7fef6010cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 --2- 172.21.15.100:0/2094208181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef60102870 0x7fef6010cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 -- 172.21.15.100:0/2094208181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef600fc9e0 msgr2=0x7fef60112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 --2- 172.21.15.100:0/2094208181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef600fc9e0 0x7fef60112dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 -- 172.21.15.100:0/2094208181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480098b0 con 0x7fef600fd390 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef5effd640 1 --2- 172.21.15.100:0/2094208181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fd390 0x7fef60113310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef48004750 tx=0x7fef48004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48038070 con 0x7fef600fd390 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fef48007b20 con 0x7fef600fd390 2024-09-10T18:36:34.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6010db60 con 0x7fef600fd390 2024-09-10T18:36:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4803c890 con 0x7fef600fd390 2024-09-10T18:36:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6006f940 con 0x7fef600fd390 2024-09-10T18:36:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef60101d00 con 0x7fef600fd390 2024-09-10T18:36:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.040+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fef48030070 con 0x7fef600fd390 2024-09-10T18:36:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef57fff640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6800/2155730476 conn(0x7fef4c0732d0 0x7fef4c075780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fef480ae9a0 con 0x7fef600fd390 2024-09-10T18:36:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef5f7fe640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6800/2155730476 conn(0x7fef4c0732d0 0x7fef4c075780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef57fff640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6801/2899184110 conn(0x7fef4c07ccc0 0x7fef4c07f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:34.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 --> v2:172.21.15.100:6801/2899184110 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef4c07f7e0 con 0x7fef4c07ccc0 2024-09-10T18:36:34.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef480b3240 con 0x7fef600fd390 2024-09-10T18:36:34.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef5ffff640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6801/2899184110 conn(0x7fef4c07ccc0 0x7fef4c07f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:34.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef5f7fe640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6800/2155730476 conn(0x7fef4c0732d0 0x7fef4c075780 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef50004a70 tx=0x7fef5000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:34.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef5ffff640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6801/2899184110 conn(0x7fef4c07ccc0 0x7fef4c07f150 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:34.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.044+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== osd.0 v2:172.21.15.100:6801/2899184110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fef4c07f7e0 con 0x7fef4c07ccc0 2024-09-10T18:36:34.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 --> v2:172.21.15.100:6801/2899184110 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef601020b0 con 0x7fef4c07ccc0 2024-09-10T18:36:34.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef57fff640 1 -- 172.21.15.100:0/2094208181 <== osd.0 v2:172.21.15.100:6801/2899184110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef601020b0 con 0x7fef4c07ccc0 2024-09-10T18:36:34.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6801/2899184110 conn(0x7fef4c07ccc0 msgr2=0x7fef4c07f150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:34.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef65c37640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6801/2899184110 conn(0x7fef4c07ccc0 0x7fef4c07f150 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6800/2155730476 conn(0x7fef4c0732d0 msgr2=0x7fef4c075780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef65c37640 1 --2- 172.21.15.100:0/2094208181 >> v2:172.21.15.100:6800/2155730476 conn(0x7fef4c0732d0 0x7fef4c075780 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef50004a70 tx=0x7fef5000b040 comp rx=0 tx=0).stop 2024-09-10T18:36:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fd390 msgr2=0x7fef60113310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef65c37640 1 --2- 172.21.15.100:0/2094208181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef600fd390 0x7fef60113310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fef48004750 tx=0x7fef48004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.072+0000 7fef5ffff640 1 -- 172.21.15.100:0/2094208181 reap_dead start 2024-09-10T18:36:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.076+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 shutdown_connections 2024-09-10T18:36:34.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.076+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 >> 172.21.15.100:0/2094208181 conn(0x7fef600747e0 msgr2=0x7fef601003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:34.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.076+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 shutdown_connections 2024-09-10T18:36:34.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:34.076+0000 7fef65c37640 1 -- 172.21.15.100:0/2094208181 wait complete. 2024-09-10T18:36:34.839 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:36:34.841 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: cluster 2024-09-10T18:36:33.262049+0000 mgr.y (mgr.24512) 85 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:34.841 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:33.839133+0000 mon.a (mon.0) 829 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:34.841 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:33.846856+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:34.842 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:34.235103+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:34.842 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:34.236575+0000 mon.c (mon.2) 73 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:34.842 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:34.247899+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:34.842 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:34.297386+0000 mon.c (mon.2) 74 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:34.842 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:34.299832+0000 mon.c (mon.2) 75 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:34.842 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:34.301188+0000 mon.c (mon.2) 76 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:34.842 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:34 smithi151 bash[22565]: audit 2024-09-10T18:36:34.311455+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: cluster 2024-09-10T18:36:33.262049+0000 mgr.y (mgr.24512) 85 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:35.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:33.839133+0000 mon.a (mon.0) 829 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:33.846856+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:34.235103+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:35.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:34.236575+0000 mon.c (mon.2) 73 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:35.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:34.247899+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:34.297386+0000 mon.c (mon.2) 74 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:34.299832+0000 mon.c (mon.2) 75 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:34.301188+0000 mon.c (mon.2) 76 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[18597]: audit 2024-09-10T18:36:34.311455+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: cluster 2024-09-10T18:36:33.262049+0000 mgr.y (mgr.24512) 85 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:33.839133+0000 mon.a (mon.0) 829 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:33.846856+0000 mon.a (mon.0) 830 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:34.235103+0000 mon.c (mon.2) 72 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:34.236575+0000 mon.c (mon.2) 73 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:35.093 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:34.247899+0000 mon.a (mon.0) 831 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.094 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:34.297386+0000 mon.c (mon.2) 74 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:35.094 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:34.299832+0000 mon.c (mon.2) 75 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:35.094 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:34.301188+0000 mon.c (mon.2) 76 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:35.094 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:34 smithi100 bash[31607]: audit 2024-09-10T18:36:34.311455+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:36:35.739 INFO:teuthology.orchestra.run.smithi100.stdout:184683593778 2024-09-10T18:36:35.739 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/1448663165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c14101780 msgr2=0x7f0c14101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 --2- 172.21.15.100:0/1448663165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c14101780 0x7f0c14101b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c04004980 tx=0x7f0c04009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/1448663165 shutdown_connections 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 --2- 172.21.15.100:0/1448663165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c140fc110 0x7f0c14108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 --2- 172.21.15.100:0/1448663165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c140fb760 0x7f0c140fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 --2- 172.21.15.100:0/1448663165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c14101780 0x7f0c14101b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/1448663165 >> 172.21.15.100:0/1448663165 conn(0x7f0c14072400 msgr2=0x7f0c140f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/1448663165 shutdown_connections 2024-09-10T18:36:35.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/1448663165 wait complete. 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 Processor -- start 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- start start 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140fb760 0x7f0c14111cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c140fc110 0x7f0c14112210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14101780 0x7f0c14065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14066270 con 0x7f0c14101780 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140663e0 con 0x7f0c140fc110 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c1a0ae640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140665a0 con 0x7f0c140fb760 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140fb760 0x7f0c14111cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140fb760 0x7f0c14111cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41340/0 (socket says 172.21.15.100:41340) 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c137fe640 1 -- 172.21.15.100:0/2641926813 learned_addr learned my addr 172.21.15.100:0/2641926813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c13fff640 1 --2- 172.21.15.100:0/2641926813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14101780 0x7f0c14065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.828+0000 7f0c12ffd640 1 --2- 172.21.15.100:0/2641926813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c140fc110 0x7f0c14112210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c137fe640 1 -- 172.21.15.100:0/2641926813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c140fc110 msgr2=0x7f0c14112210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c137fe640 1 --2- 172.21.15.100:0/2641926813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c140fc110 0x7f0c14112210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c137fe640 1 -- 172.21.15.100:0/2641926813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14101780 msgr2=0x7f0c14065bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c137fe640 1 --2- 172.21.15.100:0/2641926813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14101780 0x7f0c14065bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c137fe640 1 -- 172.21.15.100:0/2641926813 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04005040 con 0x7f0c140fb760 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c13fff640 1 --2- 172.21.15.100:0/2641926813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14101780 0x7f0c14065bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c137fe640 1 --2- 172.21.15.100:0/2641926813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140fb760 0x7f0c14111cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c04004950 tx=0x7f0c04009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0402bdb0 con 0x7f0c140fb760 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c14066820 con 0x7f0c140fb760 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1410c050 con 0x7f0c140fb760 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f0c0403c8b0 con 0x7f0c140fb760 2024-09-10T18:36:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04033650 con 0x7f0c140fb760 2024-09-10T18:36:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c00000f20 con 0x7f0c140fb760 2024-09-10T18:36:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.832+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f0c04030020 con 0x7f0c140fb760 2024-09-10T18:36:35.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0bfbfff640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0bfc06eec0 0x7f0bfc071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f0c040ae090 con 0x7f0c140fb760 2024-09-10T18:36:35.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0c12ffd640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0bfc06eec0 0x7f0bfc071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0bfbfff640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6813/2754318461 conn(0x7f0bfc0788b0 0x7f0bfc07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 --> v2:172.21.15.100:6813/2754318461 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bfc07b3d0 con 0x7f0bfc0788b0 2024-09-10T18:36:35.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c040b0720 con 0x7f0c140fb760 2024-09-10T18:36:35.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0c13fff640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6813/2754318461 conn(0x7f0bfc0788b0 0x7f0bfc07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0c13fff640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6813/2754318461 conn(0x7f0bfc0788b0 0x7f0bfc07ad40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0c12ffd640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0bfc06eec0 0x7f0bfc071370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0c1410e710 tx=0x7f0c08009040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.836+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== osd.3 v2:172.21.15.100:6813/2754318461 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0bfc07b3d0 con 0x7f0bfc0788b0 2024-09-10T18:36:35.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- 172.21.15.100:0/3137325017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9318102870 msgr2=0x7f9318102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 --2- 172.21.15.100:0/3137325017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9318102870 0x7f9318102c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9308004950 tx=0x7f9308009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- 172.21.15.100:0/3137325017 shutdown_connections 2024-09-10T18:36:35.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 --2- 172.21.15.100:0/3137325017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93180fba40 0x7f9318109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 --2- 172.21.15.100:0/3137325017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93180fb0b0 0x7f93180fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 --2- 172.21.15.100:0/3137325017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9318102870 0x7f9318102c40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- 172.21.15.100:0/3137325017 >> 172.21.15.100:0/3137325017 conn(0x7f9318071c00 msgr2=0x7f9318072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:35.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- 172.21.15.100:0/3137325017 shutdown_connections 2024-09-10T18:36:35.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- 172.21.15.100:0/3137325017 wait complete. 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 Processor -- start 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- start start 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93180fb0b0 0x7f9318112d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93180fba40 0x7f93181132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318102870 0x7f931810cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931810d460 con 0x7f9318102870 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931810d5d0 con 0x7f93180fba40 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931cf98640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931810d790 con 0x7f93180fb0b0 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93180fb0b0 0x7f9318112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f9316d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318102870 0x7f931810cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93180fb0b0 0x7f9318112d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41356/0 (socket says 172.21.15.100:41356) 2024-09-10T18:36:35.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f931659c640 1 -- 172.21.15.100:0/2745346983 learned_addr learned my addr 172.21.15.100:0/2745346983 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:35.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.848+0000 7f9315d9b640 1 --2- 172.21.15.100:0/2745346983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93180fba40 0x7f93181132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9316d9d640 1 -- 172.21.15.100:0/2745346983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93180fb0b0 msgr2=0x7f9318112d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9316d9d640 1 --2- 172.21.15.100:0/2745346983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93180fb0b0 0x7f9318112d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9316d9d640 1 -- 172.21.15.100:0/2745346983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93180fba40 msgr2=0x7f93181132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9316d9d640 1 --2- 172.21.15.100:0/2745346983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93180fba40 0x7f93181132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9316d9d640 1 -- 172.21.15.100:0/2745346983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9308005040 con 0x7f9318102870 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9316d9d640 1 --2- 172.21.15.100:0/2745346983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318102870 0x7f931810cf20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f931000db80 tx=0x7f9310004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9306ffd640 1 -- 172.21.15.100:0/2745346983 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310017070 con 0x7f9318102870 2024-09-10T18:36:35.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9306ffd640 1 -- 172.21.15.100:0/2745346983 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f931000f040 con 0x7f9318102870 2024-09-10T18:36:35.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931810d990 con 0x7f9318102870 2024-09-10T18:36:35.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9306ffd640 1 -- 172.21.15.100:0/2745346983 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310003660 con 0x7f9318102870 2024-09-10T18:36:35.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9318117bd0 con 0x7f9318102870 2024-09-10T18:36:35.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9306ffd640 1 -- 172.21.15.100:0/2745346983 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f9310003a90 con 0x7f9318102870 2024-09-10T18:36:35.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9306ffd640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6800/2155730476 conn(0x7f930006eec0 0x7f9300071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f931659c640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6800/2155730476 conn(0x7f930006eec0 0x7f9300071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.852+0000 7f9306ffd640 1 -- 172.21.15.100:0/2745346983 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f931008e440 con 0x7f9318102870 2024-09-10T18:36:35.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f931cf98640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6809/370401307 conn(0x7f9318101eb0 0x7f931805e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 --> v2:172.21.15.100:6809/370401307 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9318064ea0 con 0x7f9318101eb0 2024-09-10T18:36:35.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f9315d9b640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6809/370401307 conn(0x7f9318101eb0 0x7f931805e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f931659c640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6800/2155730476 conn(0x7f930006eec0 0x7f9300071370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9308009f80 tx=0x7f9308009da0 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f648f4ce640 1 -- 172.21.15.100:0/1509274452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6488101890 msgr2=0x7f6488101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f648f4ce640 1 --2- 172.21.15.100:0/1509274452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6488101890 0x7f6488101c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6480009cd0 tx=0x7f6480004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:35.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f9315d9b640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6809/370401307 conn(0x7f9318101eb0 0x7f931805e060 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.856+0000 7f9306ffd640 1 -- 172.21.15.100:0/2745346983 <== osd.2 v2:172.21.15.100:6809/370401307 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9318064ea0 con 0x7f9318101eb0 2024-09-10T18:36:35.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- 172.21.15.100:0/1509274452 shutdown_connections 2024-09-10T18:36:35.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 --2- 172.21.15.100:0/1509274452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64880fc030 0x7f64881085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 --2- 172.21.15.100:0/1509274452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64881021a0 0x7f64880fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 --2- 172.21.15.100:0/1509274452 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6488101890 0x7f6488101c60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- 172.21.15.100:0/1509274452 >> 172.21.15.100:0/1509274452 conn(0x7f6488072580 msgr2=0x7f6488072980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:35.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- 172.21.15.100:0/1509274452 shutdown_connections 2024-09-10T18:36:35.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- 172.21.15.100:0/1509274452 wait complete. 2024-09-10T18:36:35.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 --> v2:172.21.15.100:6813/2754318461 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c00002b30 con 0x7f0bfc0788b0 2024-09-10T18:36:35.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 Processor -- start 2024-09-10T18:36:35.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- start start 2024-09-10T18:36:35.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0bfbfff640 1 -- 172.21.15.100:0/2641926813 <== osd.3 v2:172.21.15.100:6813/2754318461 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c00002b30 con 0x7f0bfc0788b0 2024-09-10T18:36:35.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64880fc030 0x7f6488103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6488101890 0x7f64881040f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64881021a0 0x7f648811f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6488104900 con 0x7f64880fc030 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6488104a70 con 0x7f6488101890 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648f4ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648811fba0 con 0x7f64881021a0 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648da6b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64881021a0 0x7f648811f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64880fc030 0x7f6488103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64880fc030 0x7f6488103bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38304/0 (socket says 172.21.15.100:38304) 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 -- 172.21.15.100:0/2780894601 learned_addr learned my addr 172.21.15.100:0/2780894601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648ca69640 1 --2- 172.21.15.100:0/2780894601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6488101890 0x7f64881040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 -- 172.21.15.100:0/2780894601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64881021a0 msgr2=0x7f648811f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 --2- 172.21.15.100:0/2780894601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64881021a0 0x7f648811f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 -- 172.21.15.100:0/2780894601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6488101890 msgr2=0x7f64881040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 --2- 172.21.15.100:0/2780894601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6488101890 0x7f64881040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 -- 172.21.15.100:0/2780894601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800098b0 con 0x7f64880fc030 2024-09-10T18:36:35.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6813/2754318461 conn(0x7f0bfc0788b0 msgr2=0x7f0bfc07ad40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6813/2754318461 conn(0x7f0bfc0788b0 0x7f0bfc07ad40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f648d26a640 1 --2- 172.21.15.100:0/2780894601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64880fc030 0x7f6488103bb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6480032f50 tx=0x7f6480004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0bfc06eec0 msgr2=0x7f0bfc071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 --2- 172.21.15.100:0/2641926813 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0bfc06eec0 0x7f0bfc071370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0c1410e710 tx=0x7f0c08009040 comp rx=0 tx=0).stop 2024-09-10T18:36:35.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140fb760 msgr2=0x7f0c14111cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 --2- 172.21.15.100:0/2641926813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140fb760 0x7f0c14111cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c04004950 tx=0x7f0c04009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480007c20 con 0x7f64880fc030 2024-09-10T18:36:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c13fff640 1 -- 172.21.15.100:0/2641926813 reap_dead start 2024-09-10T18:36:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f6480007d80 con 0x7f64880fc030 2024-09-10T18:36:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 shutdown_connections 2024-09-10T18:36:35.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 >> 172.21.15.100:0/2641926813 conn(0x7f0c14072400 msgr2=0x7f0c140ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 shutdown_connections 2024-09-10T18:36:35.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.860+0000 7f0c1a0ae640 1 -- 172.21.15.100:0/2641926813 wait complete. 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648003d900 con 0x7f64880fc030 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648811fe20 con 0x7f64880fc030 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6488120260 con 0x7f64880fc030 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64880feb90 con 0x7f64880fc030 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f648002f030 con 0x7f64880fc030 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f647dffb640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.100:6800/2155730476 conn(0x7f647406efe0 0x7f6474071490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f64800af240 con 0x7f64880fc030 2024-09-10T18:36:35.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f648ca69640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.100:6800/2155730476 conn(0x7f647406efe0 0x7f6474071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f647dffb640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.151:6804/3330203218 conn(0x7f6474078a50 0x7f647407aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 --> v2:172.21.15.151:6804/3330203218 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f647407b570 con 0x7f6474078a50 2024-09-10T18:36:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64800b18d0 con 0x7f64880fc030 2024-09-10T18:36:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.864+0000 7f648da6b640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.151:6804/3330203218 conn(0x7f6474078a50 0x7f647407aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.868+0000 7f648ca69640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.100:6800/2155730476 conn(0x7f647406efe0 0x7f6474071490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6488105290 tx=0x7f6484009120 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.868+0000 7f648da6b640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.151:6804/3330203218 conn(0x7f6474078a50 0x7f647407aee0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:35.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.868+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== osd.5 v2:172.21.15.151:6804/3330203218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f647407b570 con 0x7f6474078a50 2024-09-10T18:36:35.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 --> v2:172.21.15.151:6804/3330203218 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64880fcf30 con 0x7f6474078a50 2024-09-10T18:36:35.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 --> v2:172.21.15.100:6809/370401307 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f931804bf60 con 0x7f9318101eb0 2024-09-10T18:36:35.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f647dffb640 1 -- 172.21.15.100:0/2780894601 <== osd.5 v2:172.21.15.151:6804/3330203218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f64880fcf30 con 0x7f6474078a50 2024-09-10T18:36:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f9306ffd640 1 -- 172.21.15.100:0/2745346983 <== osd.2 v2:172.21.15.100:6809/370401307 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f931804bf60 con 0x7f9318101eb0 2024-09-10T18:36:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 >> v2:172.21.15.151:6804/3330203218 conn(0x7f6474078a50 msgr2=0x7f647407aee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.151:6804/3330203218 conn(0x7f6474078a50 0x7f647407aee0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 >> v2:172.21.15.100:6800/2155730476 conn(0x7f647406efe0 msgr2=0x7f6474071490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 --2- 172.21.15.100:0/2780894601 >> v2:172.21.15.100:6800/2155730476 conn(0x7f647406efe0 0x7f6474071490 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6488105290 tx=0x7f6484009120 comp rx=0 tx=0).stop 2024-09-10T18:36:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64880fc030 msgr2=0x7f6488103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 --2- 172.21.15.100:0/2780894601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64880fc030 0x7f6488103bb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6480032f50 tx=0x7f6480004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648da6b640 1 -- 172.21.15.100:0/2780894601 reap_dead start 2024-09-10T18:36:35.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6809/370401307 conn(0x7f9318101eb0 msgr2=0x7f931805e060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6809/370401307 conn(0x7f9318101eb0 0x7f931805e060 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 shutdown_connections 2024-09-10T18:36:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 >> 172.21.15.100:0/2780894601 conn(0x7f6488072580 msgr2=0x7f64880fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6800/2155730476 conn(0x7f930006eec0 msgr2=0x7f9300071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 --2- 172.21.15.100:0/2745346983 >> v2:172.21.15.100:6800/2155730476 conn(0x7f930006eec0 0x7f9300071370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9308009f80 tx=0x7f9308009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318102870 msgr2=0x7f931810cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 --2- 172.21.15.100:0/2745346983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318102870 0x7f931810cf20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f931000db80 tx=0x7f9310004db0 comp rx=0 tx=0).stop 2024-09-10T18:36:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f9316d9d640 1 -- 172.21.15.100:0/2745346983 reap_dead start 2024-09-10T18:36:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 shutdown_connections 2024-09-10T18:36:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 shutdown_connections 2024-09-10T18:36:35.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f648f4ce640 1 -- 172.21.15.100:0/2780894601 wait complete. 2024-09-10T18:36:35.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 >> 172.21.15.100:0/2745346983 conn(0x7f9318071c00 msgr2=0x7f93181003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:35.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 shutdown_connections 2024-09-10T18:36:35.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:35.884+0000 7f931cf98640 1 -- 172.21.15.100:0/2745346983 wait complete. 2024-09-10T18:36:35.989 INFO:teuthology.orchestra.run.smithi100.stdout:55834574954 2024-09-10T18:36:35.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T18:36:35.991 INFO:teuthology.orchestra.run.smithi100.stdout:34359738483 2024-09-10T18:36:35.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T18:36:37.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:36 smithi151 bash[22565]: cluster 2024-09-10T18:36:35.263447+0000 mgr.y (mgr.24512) 86 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:36 smithi100 bash[18597]: cluster 2024-09-10T18:36:35.263447+0000 mgr.y (mgr.24512) 86 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:36 smithi100 bash[31607]: cluster 2024-09-10T18:36:35.263447+0000 mgr.y (mgr.24512) 86 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:37.558 INFO:teuthology.orchestra.run.smithi100.stdout:158913790015 2024-09-10T18:36:37.558 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T18:36:37.603 INFO:teuthology.orchestra.run.smithi100.stdout:111669149781 2024-09-10T18:36:37.603 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T18:36:37.645 INFO:teuthology.orchestra.run.smithi100.stdout:77309411424 2024-09-10T18:36:37.645 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T18:36:39.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:38 smithi151 bash[22565]: cluster 2024-09-10T18:36:37.264501+0000 mgr.y (mgr.24512) 87 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:39.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:38 smithi100 bash[18597]: cluster 2024-09-10T18:36:37.264501+0000 mgr.y (mgr.24512) 87 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:38 smithi100 bash[31607]: cluster 2024-09-10T18:36:37.264501+0000 mgr.y (mgr.24512) 87 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:40.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:39 smithi151 bash[22565]: audit 2024-09-10T18:36:39.315647+0000 mon.c (mon.2) 77 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:40.259 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:39 smithi100 bash[18597]: audit 2024-09-10T18:36:39.315647+0000 mon.c (mon.2) 77 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:40.260 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:39 smithi100 bash[31607]: audit 2024-09-10T18:36:39.315647+0000 mon.c (mon.2) 77 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:41.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:40 smithi151 bash[22565]: cluster 2024-09-10T18:36:39.265259+0000 mgr.y (mgr.24512) 88 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:41.177 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:41.177 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:41.178 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:41.178 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:41.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:40 smithi100 bash[31607]: cluster 2024-09-10T18:36:39.265259+0000 mgr.y (mgr.24512) 88 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:41.193 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:40 smithi100 bash[18597]: cluster 2024-09-10T18:36:39.265259+0000 mgr.y (mgr.24512) 88 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:41.450 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:41] "GET /metrics HTTP/1.1" 200 219700 "" "Prometheus/2.43.0" 2024-09-10T18:36:42.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:36:41 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:36:43.170 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:42 smithi100 bash[18597]: cluster 2024-09-10T18:36:41.266419+0000 mgr.y (mgr.24512) 89 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:43.171 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:42 smithi100 bash[18597]: audit 2024-09-10T18:36:41.958905+0000 mgr.y (mgr.24512) 90 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:43.171 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:42 smithi100 bash[31607]: cluster 2024-09-10T18:36:41.266419+0000 mgr.y (mgr.24512) 89 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:43.171 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:42 smithi100 bash[31607]: audit 2024-09-10T18:36:41.958905+0000 mgr.y (mgr.24512) 90 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:43.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:42 smithi151 bash[22565]: cluster 2024-09-10T18:36:41.266419+0000 mgr.y (mgr.24512) 89 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:43.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:42 smithi151 bash[22565]: audit 2024-09-10T18:36:41.958905+0000 mgr.y (mgr.24512) 90 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:43.411 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:43.411 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:43.412 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:44.870 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:36:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:36:45.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:44 smithi151 bash[22565]: cluster 2024-09-10T18:36:43.267166+0000 mgr.y (mgr.24512) 91 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:44 smithi100 bash[18597]: cluster 2024-09-10T18:36:43.267166+0000 mgr.y (mgr.24512) 91 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:45.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:44 smithi100 bash[31607]: cluster 2024-09-10T18:36:43.267166+0000 mgr.y (mgr.24512) 91 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:46.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.291+0000 7f4f88671640 1 -- 172.21.15.100:0/1426600598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80101b80 msgr2=0x7f4f80066490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:46.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.291+0000 7f4f88671640 1 --2- 172.21.15.100:0/1426600598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80101b80 0x7f4f80066490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4f70004890 tx=0x7f4f70009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.291+0000 7f4f88671640 1 -- 172.21.15.100:0/1426600598 shutdown_connections 2024-09-10T18:36:46.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.291+0000 7f4f88671640 1 --2- 172.21.15.100:0/1426600598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f80066b70 0x7f4f800ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.291+0000 7f4f88671640 1 --2- 172.21.15.100:0/1426600598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80101b80 0x7f4f80066490 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.291+0000 7f4f88671640 1 --2- 172.21.15.100:0/1426600598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f801017b0 0x7f4f80065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.291+0000 7f4f88671640 1 -- 172.21.15.100:0/1426600598 >> 172.21.15.100:0/1426600598 conn(0x7f4f80072430 msgr2=0x7f4f80072830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:46.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- 172.21.15.100:0/1426600598 shutdown_connections 2024-09-10T18:36:46.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- 172.21.15.100:0/1426600598 wait complete. 2024-09-10T18:36:46.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 Processor -- start 2024-09-10T18:36:46.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- start start 2024-09-10T18:36:46.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80066b70 0x7f4f801193d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f801017b0 0x7f4f80119910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f80101b80 0x7f4f80113560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80113bb0 con 0x7f4f80066b70 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80113d20 con 0x7f4f80101b80 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80113ee0 con 0x7f4f801017b0 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80066b70 0x7f4f801193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80066b70 0x7f4f801193d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45954/0 (socket says 172.21.15.100:45954) 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 -- 172.21.15.100:0/4164752174 learned_addr learned my addr 172.21.15.100:0/4164752174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f85c0c640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f801017b0 0x7f4f80119910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f86c0e640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f80101b80 0x7f4f80113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 -- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f801017b0 msgr2=0x7f4f80119910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f801017b0 0x7f4f80119910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 -- 172.21.15.100:0/4164752174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f80101b80 msgr2=0x7f4f80113560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f80101b80 0x7f4f80113560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 -- 172.21.15.100:0/4164752174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f70005040 con 0x7f4f80066b70 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f86c0e640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f80101b80 0x7f4f80113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f85c0c640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f801017b0 0x7f4f80119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f8640d640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80066b70 0x7f4f801193d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00ebc0 tx=0x7f4f7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f76ffd640 1 -- 172.21.15.100:0/4164752174 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c007c30 con 0x7f4f80066b70 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f76ffd640 1 -- 172.21.15.100:0/4164752174 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f4f7c00f040 con 0x7f4f80066b70 2024-09-10T18:36:46.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f76ffd640 1 -- 172.21.15.100:0/4164752174 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c009340 con 0x7f4f80066b70 2024-09-10T18:36:46.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f801141c0 con 0x7f4f80066b70 2024-09-10T18:36:46.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.295+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801203c0 con 0x7f4f80066b70 2024-09-10T18:36:46.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.299+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f800fbe00 con 0x7f4f80066b70 2024-09-10T18:36:46.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.299+0000 7f4f76ffd640 1 -- 172.21.15.100:0/4164752174 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f4f7c009730 con 0x7f4f80066b70 2024-09-10T18:36:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.299+0000 7f4f76ffd640 1 --2- 172.21.15.100:0/4164752174 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4f6c06eec0 0x7f4f6c071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.299+0000 7f4f76ffd640 1 -- 172.21.15.100:0/4164752174 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f4f7c08e1f0 con 0x7f4f80066b70 2024-09-10T18:36:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.299+0000 7f4f85c0c640 1 --2- 172.21.15.100:0/4164752174 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4f6c06eec0 0x7f4f6c071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.299+0000 7f4f85c0c640 1 --2- 172.21.15.100:0/4164752174 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4f6c06eec0 0x7f4f6c071370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f800fc650 tx=0x7f4f70035040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.307+0000 7f4f76ffd640 1 -- 172.21.15.100:0/4164752174 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f4f7c04e2a0 con 0x7f4f80066b70 2024-09-10T18:36:46.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4f801123c0 con 0x7f4f80066b70 2024-09-10T18:36:46.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f76ffd640 1 -- 172.21.15.100:0/4164752174 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f7c05b090 con 0x7f4f80066b70 2024-09-10T18:36:46.485 INFO:teuthology.orchestra.run.smithi100.stdout:210453397546 2024-09-10T18:36:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4f6c06eec0 msgr2=0x7f4f6c071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:46.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 --2- 172.21.15.100:0/4164752174 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4f6c06eec0 0x7f4f6c071370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4f800fc650 tx=0x7f4f70035040 comp rx=0 tx=0).stop 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80066b70 msgr2=0x7f4f801193d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80066b70 0x7f4f801193d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00ebc0 tx=0x7f4f7c00b040 comp rx=0 tx=0).stop 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 shutdown_connections 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 --2- 172.21.15.100:0/4164752174 >> v2:172.21.15.100:6800/2155730476 conn(0x7f4f6c06eec0 0x7f4f6c071370 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f80101b80 0x7f4f80113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f801017b0 0x7f4f80119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 --2- 172.21.15.100:0/4164752174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f80066b70 0x7f4f801193d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 >> 172.21.15.100:0/4164752174 conn(0x7f4f80072430 msgr2=0x7f4f800fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.483+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 shutdown_connections 2024-09-10T18:36:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:46.487+0000 7f4f88671640 1 -- 172.21.15.100:0/4164752174 wait complete. 2024-09-10T18:36:47.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:46 smithi151 bash[22565]: cluster 2024-09-10T18:36:45.268619+0000 mgr.y (mgr.24512) 92 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:47.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:46 smithi151 bash[22565]: audit 2024-09-10T18:36:46.486294+0000 mon.a (mon.0) 833 : audit [DBG] from='client.? 172.21.15.100:0/4164752174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:36:47.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:46 smithi100 bash[31607]: cluster 2024-09-10T18:36:45.268619+0000 mgr.y (mgr.24512) 92 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:47.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:46 smithi100 bash[31607]: audit 2024-09-10T18:36:46.486294+0000 mon.a (mon.0) 833 : audit [DBG] from='client.? 172.21.15.100:0/4164752174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:36:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:46 smithi100 bash[18597]: cluster 2024-09-10T18:36:45.268619+0000 mgr.y (mgr.24512) 92 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:46 smithi100 bash[18597]: audit 2024-09-10T18:36:46.486294+0000 mon.a (mon.0) 833 : audit [DBG] from='client.? 172.21.15.100:0/4164752174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:36:49.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:48 smithi151 bash[22565]: cluster 2024-09-10T18:36:47.269309+0000 mgr.y (mgr.24512) 93 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:49.192 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:48 smithi100 bash[18597]: cluster 2024-09-10T18:36:47.269309+0000 mgr.y (mgr.24512) 93 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:49.192 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:48 smithi100 bash[31607]: cluster 2024-09-10T18:36:47.269309+0000 mgr.y (mgr.24512) 93 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:51.173 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:50 smithi100 bash[18597]: cluster 2024-09-10T18:36:49.270014+0000 mgr.y (mgr.24512) 94 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:51.173 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:50 smithi100 bash[31607]: cluster 2024-09-10T18:36:49.270014+0000 mgr.y (mgr.24512) 94 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:51.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:50 smithi151 bash[22565]: cluster 2024-09-10T18:36:49.270014+0000 mgr.y (mgr.24512) 94 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:51.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397542 got 210453397546 for osd.7 2024-09-10T18:36:51.342 DEBUG:teuthology.parallel:result is None 2024-09-10T18:36:51.570 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:36:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:51] "GET /metrics HTTP/1.1" 200 219700 "" "Prometheus/2.43.0" 2024-09-10T18:36:52.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:36:51 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:36:53.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:52 smithi100 bash[18597]: cluster 2024-09-10T18:36:51.271520+0000 mgr.y (mgr.24512) 95 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:53.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:52 smithi100 bash[18597]: audit 2024-09-10T18:36:51.965063+0000 mgr.y (mgr.24512) 96 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:53.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:52 smithi100 bash[31607]: cluster 2024-09-10T18:36:51.271520+0000 mgr.y (mgr.24512) 95 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:53.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:52 smithi100 bash[31607]: audit 2024-09-10T18:36:51.965063+0000 mgr.y (mgr.24512) 96 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:53.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:52 smithi151 bash[22565]: cluster 2024-09-10T18:36:51.271520+0000 mgr.y (mgr.24512) 95 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:53.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:52 smithi151 bash[22565]: audit 2024-09-10T18:36:51.965063+0000 mgr.y (mgr.24512) 96 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:36:54.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- 172.21.15.100:0/2494928324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 msgr2=0x7f0bc0108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 --2- 172.21.15.100:0/2494928324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 0x7f0bc0108510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009d00 tx=0x7f0bb8004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- 172.21.15.100:0/2494928324 shutdown_connections 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 --2- 172.21.15.100:0/2494928324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 0x7f0bc0108510 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 --2- 172.21.15.100:0/2494928324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 0x7f0bc00fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 --2- 172.21.15.100:0/2494928324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc0101760 0x7f0bc0101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- 172.21.15.100:0/2494928324 >> 172.21.15.100:0/2494928324 conn(0x7f0bc0072440 msgr2=0x7f0bc0072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- 172.21.15.100:0/2494928324 shutdown_connections 2024-09-10T18:36:54.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- 172.21.15.100:0/2494928324 wait complete. 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 Processor -- start 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- start start 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 0x7f0bc0112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 0x7f0bc0113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc0101760 0x7f0bc0065300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0065950 con 0x7f0bc00fc300 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0065ac0 con 0x7f0bc0101760 2024-09-10T18:36:54.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc683a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0065c80 con 0x7f0bc00fb950 2024-09-10T18:36:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 0x7f0bc0112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 0x7f0bc0113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 0x7f0bc0112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:39632/0 (socket says 172.21.15.100:39632) 2024-09-10T18:36:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 -- 172.21.15.100:0/1809723740 learned_addr learned my addr 172.21.15.100:0/1809723740 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bc4dd7640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc0101760 0x7f0bc0065300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 -- 172.21.15.100:0/1809723740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc0101760 msgr2=0x7f0bc0065300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:54.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc0101760 0x7f0bc0065300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 -- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 msgr2=0x7f0bc0113120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 0x7f0bc0113120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 -- 172.21.15.100:0/1809723740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb80098b0 con 0x7f0bc00fb950 2024-09-10T18:36:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbf7fe640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 0x7f0bc0113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:36:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.851+0000 7f0bbffff640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 0x7f0bc0112be0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c8b0 tx=0x7f0bb0004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bbcff9640 1 -- 172.21.15.100:0/1809723740 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000a5c0 con 0x7f0bc00fb950 2024-09-10T18:36:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bbcff9640 1 -- 172.21.15.100:0/1809723740 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f0bb000ab90 con 0x7f0bc00fb950 2024-09-10T18:36:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc0065f60 con 0x7f0bc00fb950 2024-09-10T18:36:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bbcff9640 1 -- 172.21.15.100:0/1809723740 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0012ac0 con 0x7f0bc00fb950 2024-09-10T18:36:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc0119ba0 con 0x7f0bc00fb950 2024-09-10T18:36:54.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc00fce80 con 0x7f0bc00fb950 2024-09-10T18:36:54.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bbcff9640 1 -- 172.21.15.100:0/1809723740 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f0bb000a720 con 0x7f0bc00fb950 2024-09-10T18:36:54.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.855+0000 7f0bbcff9640 1 --2- 172.21.15.100:0/1809723740 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0ba806eec0 0x7f0ba8071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:54.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.859+0000 7f0bbf7fe640 1 --2- 172.21.15.100:0/1809723740 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0ba806eec0 0x7f0ba8071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:54.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.859+0000 7f0bbcff9640 1 -- 172.21.15.100:0/1809723740 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f0bb008e640 con 0x7f0bc00fb950 2024-09-10T18:36:54.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.859+0000 7f0bbf7fe640 1 --2- 172.21.15.100:0/1809723740 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0ba806eec0 0x7f0ba8071370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4004fd0 tx=0x7f0bb400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:54.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:54.863+0000 7f0bbcff9640 1 -- 172.21.15.100:0/1809723740 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f0bb005b4b0 con 0x7f0bc00fb950 2024-09-10T18:36:54.893 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:36:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:36:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:36:55.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.039+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0bc010ab50 con 0x7f0bc00fb950 2024-09-10T18:36:55.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.039+0000 7f0bbcff9640 1 -- 172.21.15.100:0/1809723740 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0bb005ae50 con 0x7f0bc00fb950 2024-09-10T18:36:55.043 INFO:teuthology.orchestra.run.smithi100.stdout:133143986253 2024-09-10T18:36:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0ba806eec0 msgr2=0x7f0ba8071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 --2- 172.21.15.100:0/1809723740 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0ba806eec0 0x7f0ba8071370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4004fd0 tx=0x7f0bb400b040 comp rx=0 tx=0).stop 2024-09-10T18:36:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 msgr2=0x7f0bc0112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:55.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 0x7f0bc0112be0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c8b0 tx=0x7f0bb0004430 comp rx=0 tx=0).stop 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 shutdown_connections 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 --2- 172.21.15.100:0/1809723740 >> v2:172.21.15.100:6800/2155730476 conn(0x7f0ba806eec0 0x7f0ba8071370 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc0101760 0x7f0bc0065300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bc00fc300 0x7f0bc0113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 --2- 172.21.15.100:0/1809723740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bc00fb950 0x7f0bc0112be0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 >> 172.21.15.100:0/1809723740 conn(0x7f0bc0072440 msgr2=0x7f0bc00fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 shutdown_connections 2024-09-10T18:36:55.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:55.047+0000 7f0bc683a640 1 -- 172.21.15.100:0/1809723740 wait complete. 2024-09-10T18:36:55.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:54 smithi151 bash[22565]: cluster 2024-09-10T18:36:53.272192+0000 mgr.y (mgr.24512) 97 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:55.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:54 smithi151 bash[22565]: audit 2024-09-10T18:36:54.316152+0000 mon.c (mon.2) 78 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:54 smithi100 bash[31607]: cluster 2024-09-10T18:36:53.272192+0000 mgr.y (mgr.24512) 97 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:54 smithi100 bash[31607]: audit 2024-09-10T18:36:54.316152+0000 mon.c (mon.2) 78 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:54 smithi100 bash[18597]: cluster 2024-09-10T18:36:53.272192+0000 mgr.y (mgr.24512) 97 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:54 smithi100 bash[18597]: audit 2024-09-10T18:36:54.316152+0000 mon.c (mon.2) 78 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:36:56.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 -- 172.21.15.100:0/1523842214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c102850 msgr2=0x7f303c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:56.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 --2- 172.21.15.100:0/1523842214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c102850 0x7f303c102c20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f302c004950 tx=0x7f302c009da0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 -- 172.21.15.100:0/1523842214 shutdown_connections 2024-09-10T18:36:56.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 --2- 172.21.15.100:0/1523842214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f303c066980 0x7f303c109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 --2- 172.21.15.100:0/1523842214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c065fd0 0x7f303c066440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 --2- 172.21.15.100:0/1523842214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c102850 0x7f303c102c20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 -- 172.21.15.100:0/1523842214 >> 172.21.15.100:0/1523842214 conn(0x7f303c0724b0 msgr2=0x7f303c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:56.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 -- 172.21.15.100:0/1523842214 shutdown_connections 2024-09-10T18:36:56.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.007+0000 7f3041ae7640 1 -- 172.21.15.100:0/1523842214 wait complete. 2024-09-10T18:36:56.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 Processor -- start 2024-09-10T18:36:56.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 -- start start 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c065fd0 0x7f303c112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c066980 0x7f303c1132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f303c102850 0x7f303c10cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c10d570 con 0x7f303c065fd0 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303c10d6e0 con 0x7f303c066980 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303c10d8a0 con 0x7f303c102850 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c065fd0 0x7f303c112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c065fd0 0x7f303c112d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57702/0 (socket says 172.21.15.100:57702) 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 -- 172.21.15.100:0/656778158 learned_addr learned my addr 172.21.15.100:0/656778158 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303bfff640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f303c102850 0x7f303c10cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303affd640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c066980 0x7f303c1132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:56.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 -- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f303c102850 msgr2=0x7f303c10cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f303c102850 0x7f303c10cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 -- 172.21.15.100:0/656778158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c066980 msgr2=0x7f303c1132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c066980 0x7f303c1132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 -- 172.21.15.100:0/656778158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c005040 con 0x7f303c065fd0 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f303b7fe640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c065fd0 0x7f303c112d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f302c02d040 tx=0x7f302c035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3033fff640 1 -- 172.21.15.100:0/656778158 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c02a960 con 0x7f303c065fd0 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3033fff640 1 -- 172.21.15.100:0/656778158 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f302c0325e0 con 0x7f303c065fd0 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303c10db20 con 0x7f303c065fd0 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3033fff640 1 -- 172.21.15.100:0/656778158 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c033690 con 0x7f303c065fd0 2024-09-10T18:36:56.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303c119db0 con 0x7f303c065fd0 2024-09-10T18:36:56.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.011+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c005010 con 0x7f303c065fd0 2024-09-10T18:36:56.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.015+0000 7f3033fff640 1 -- 172.21.15.100:0/656778158 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f302c030020 con 0x7f303c065fd0 2024-09-10T18:36:56.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.015+0000 7f3033fff640 1 --2- 172.21.15.100:0/656778158 >> v2:172.21.15.100:6800/2155730476 conn(0x7f302806eec0 0x7f3028071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:56.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.015+0000 7f3033fff640 1 -- 172.21.15.100:0/656778158 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f302c0ae610 con 0x7f303c065fd0 2024-09-10T18:36:56.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.015+0000 7f303affd640 1 --2- 172.21.15.100:0/656778158 >> v2:172.21.15.100:6800/2155730476 conn(0x7f302806eec0 0x7f3028071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:56.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.015+0000 7f303affd640 1 --2- 172.21.15.100:0/656778158 >> v2:172.21.15.100:6800/2155730476 conn(0x7f302806eec0 0x7f3028071370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f303c10e680 tx=0x7f303400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:56.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.023+0000 7f3033fff640 1 -- 172.21.15.100:0/656778158 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f302c07b540 con 0x7f303c065fd0 2024-09-10T18:36:56.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:55 smithi151 bash[22565]: audit 2024-09-10T18:36:55.044259+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? 172.21.15.100:0/1809723740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:36:56.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f301c005590 con 0x7f303c065fd0 2024-09-10T18:36:56.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3033fff640 1 -- 172.21.15.100:0/656778158 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f302c07aee0 con 0x7f303c065fd0 2024-09-10T18:36:56.205 INFO:teuthology.orchestra.run.smithi100.stdout:184683593783 2024-09-10T18:36:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 >> v2:172.21.15.100:6800/2155730476 conn(0x7f302806eec0 msgr2=0x7f3028071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 --2- 172.21.15.100:0/656778158 >> v2:172.21.15.100:6800/2155730476 conn(0x7f302806eec0 0x7f3028071370 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f303c10e680 tx=0x7f303400b040 comp rx=0 tx=0).stop 2024-09-10T18:36:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c065fd0 msgr2=0x7f303c112d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:56.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c065fd0 0x7f303c112d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f302c02d040 tx=0x7f302c035040 comp rx=0 tx=0).stop 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 shutdown_connections 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 --2- 172.21.15.100:0/656778158 >> v2:172.21.15.100:6800/2155730476 conn(0x7f302806eec0 0x7f3028071370 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f303c102850 0x7f303c10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f303c066980 0x7f303c1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 --2- 172.21.15.100:0/656778158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f303c065fd0 0x7f303c112d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 >> 172.21.15.100:0/656778158 conn(0x7f303c0724b0 msgr2=0x7f303c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 shutdown_connections 2024-09-10T18:36:56.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:56.203+0000 7f3041ae7640 1 -- 172.21.15.100:0/656778158 wait complete. 2024-09-10T18:36:56.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:55 smithi100 bash[18597]: audit 2024-09-10T18:36:55.044259+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? 172.21.15.100:0/1809723740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:36:56.231 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:55 smithi100 bash[31607]: audit 2024-09-10T18:36:55.044259+0000 mon.c (mon.2) 79 : audit [DBG] from='client.? 172.21.15.100:0/1809723740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:36:56.966 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986249 got 133143986253 for osd.4 2024-09-10T18:36:56.966 DEBUG:teuthology.parallel:result is None 2024-09-10T18:36:57.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:56 smithi100 bash[18597]: cluster 2024-09-10T18:36:55.273554+0000 mgr.y (mgr.24512) 98 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:57.164 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:56 smithi100 bash[18597]: audit 2024-09-10T18:36:56.205978+0000 mon.a (mon.0) 834 : audit [DBG] from='client.? 172.21.15.100:0/656778158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:36:57.164 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:56 smithi100 bash[31607]: cluster 2024-09-10T18:36:55.273554+0000 mgr.y (mgr.24512) 98 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:57.164 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:56 smithi100 bash[31607]: audit 2024-09-10T18:36:56.205978+0000 mon.a (mon.0) 834 : audit [DBG] from='client.? 172.21.15.100:0/656778158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:36:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 -- 172.21.15.100:0/1630415591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200ff3f0 msgr2=0x7f822010b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 --2- 172.21.15.100:0/1630415591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200ff3f0 0x7f822010b810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8218009d00 tx=0x7f8218004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 -- 172.21.15.100:0/1630415591 shutdown_connections 2024-09-10T18:36:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 --2- 172.21.15.100:0/1630415591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200ff3f0 0x7f822010b810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 --2- 172.21.15.100:0/1630415591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82200fea40 0x7f82200feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:56 smithi151 bash[22565]: cluster 2024-09-10T18:36:55.273554+0000 mgr.y (mgr.24512) 98 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:36:57.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:56 smithi151 bash[22565]: audit 2024-09-10T18:36:56.205978+0000 mon.a (mon.0) 834 : audit [DBG] from='client.? 172.21.15.100:0/656778158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 --2- 172.21.15.100:0/1630415591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220104a60 0x7f8220104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 -- 172.21.15.100:0/1630415591 >> 172.21.15.100:0/1630415591 conn(0x7f82200fa5e0 msgr2=0x7f82200fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 -- 172.21.15.100:0/1630415591 shutdown_connections 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.171+0000 7f82253ad640 1 -- 172.21.15.100:0/1630415591 wait complete. 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 Processor -- start 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 -- start start 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200fea40 0x7f8220113be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82200ff3f0 0x7f8220114120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220104a60 0x7f822010ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010f5e0 con 0x7f82200fea40 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010f750 con 0x7f8220104a60 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822010f910 con 0x7f82200ff3f0 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f821effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200fea40 0x7f8220113be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f8217fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82200ff3f0 0x7f8220114120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f821effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200fea40 0x7f8220113be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57724/0 (socket says 172.21.15.100:57724) 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f821effd640 1 -- 172.21.15.100:0/2545336604 learned_addr learned my addr 172.21.15.100:0/2545336604 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f821f7fe640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220104a60 0x7f822010ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f8217fff640 1 -- 172.21.15.100:0/2545336604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220104a60 msgr2=0x7f822010ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f8217fff640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220104a60 0x7f822010ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f8217fff640 1 -- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200fea40 msgr2=0x7f8220113be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f8217fff640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200fea40 0x7f8220113be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f8217fff640 1 -- 172.21.15.100:0/2545336604 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82180098b0 con 0x7f82200ff3f0 2024-09-10T18:36:57.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f821effd640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200fea40 0x7f8220113be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:36:57.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f8217fff640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82200ff3f0 0x7f8220114120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f820800e990 tx=0x7f820800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:57.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82177fe640 1 -- 172.21.15.100:0/2545336604 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208007c30 con 0x7f82200ff3f0 2024-09-10T18:36:57.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82177fe640 1 -- 172.21.15.100:0/2545336604 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f820800f040 con 0x7f82200ff3f0 2024-09-10T18:36:57.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822010fb90 con 0x7f82200ff3f0 2024-09-10T18:36:57.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82177fe640 1 -- 172.21.15.100:0/2545336604 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208009340 con 0x7f82200ff3f0 2024-09-10T18:36:57.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822011bd20 con 0x7f82200ff3f0 2024-09-10T18:36:57.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.175+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8220100180 con 0x7f82200ff3f0 2024-09-10T18:36:57.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.179+0000 7f82177fe640 1 -- 172.21.15.100:0/2545336604 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f8208009770 con 0x7f82200ff3f0 2024-09-10T18:36:57.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.179+0000 7f82177fe640 1 --2- 172.21.15.100:0/2545336604 >> v2:172.21.15.100:6800/2155730476 conn(0x7f820c06edb0 0x7f820c071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.179+0000 7f82177fe640 1 -- 172.21.15.100:0/2545336604 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f820808dfb0 con 0x7f82200ff3f0 2024-09-10T18:36:57.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.179+0000 7f821effd640 1 --2- 172.21.15.100:0/2545336604 >> v2:172.21.15.100:6800/2155730476 conn(0x7f820c06edb0 0x7f820c071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.179+0000 7f821effd640 1 --2- 172.21.15.100:0/2545336604 >> v2:172.21.15.100:6800/2155730476 conn(0x7f820c06edb0 0x7f820c071260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82100115b0 tx=0x7f821000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:57.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.183+0000 7f82177fe640 1 -- 172.21.15.100:0/2545336604 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f820805ae30 con 0x7f82200ff3f0 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- 172.21.15.100:0/2771173909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014066500 msgr2=0x7f1014109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 --2- 172.21.15.100:0/2771173909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014066500 0x7f1014109600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f100c009cd0 tx=0x7f100c004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- 172.21.15.100:0/2771173909 shutdown_connections 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 --2- 172.21.15.100:0/2771173909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014066500 0x7f1014109600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 --2- 172.21.15.100:0/2771173909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 0x7f1014065fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 --2- 172.21.15.100:0/2771173909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1014102850 0x7f1014102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- 172.21.15.100:0/2771173909 >> 172.21.15.100:0/2771173909 conn(0x7f1014072470 msgr2=0x7f1014072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- 172.21.15.100:0/2771173909 shutdown_connections 2024-09-10T18:36:57.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- 172.21.15.100:0/2771173909 wait complete. 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 Processor -- start 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- start start 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 0x7f10140fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1014066500 0x7f10140fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014102850 0x7f10140fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101411b130 con 0x7f1014065b50 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101411b2a0 con 0x7f1014066500 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101411b460 con 0x7f1014102850 2024-09-10T18:36:57.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 0x7f10140fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 0x7f10140fba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57730/0 (socket says 172.21.15.100:57730) 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 -- 172.21.15.100:0/170032257 learned_addr learned my addr 172.21.15.100:0/170032257 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10137fe640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014102850 0x7f10140fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f100bfff640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1014066500 0x7f10140fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 -- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014102850 msgr2=0x7f10140fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014102850 0x7f10140fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 -- 172.21.15.100:0/170032257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1014066500 msgr2=0x7f10140fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1014066500 0x7f10140fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 -- 172.21.15.100:0/170032257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc005040 con 0x7f1014065b50 2024-09-10T18:36:57.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f1012ffd640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 0x7f10140fba60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f100400ebf0 tx=0x7f100400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:57.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f100b7fe640 1 -- 172.21.15.100:0/170032257 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004007c30 con 0x7f1014065b50 2024-09-10T18:36:57.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- 172.21.15.100:0/170032257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c0098b0 con 0x7f1014065b50 2024-09-10T18:36:57.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f100b7fe640 1 -- 172.21.15.100:0/170032257 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f100400f040 con 0x7f1014065b50 2024-09-10T18:36:57.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f100b7fe640 1 -- 172.21.15.100:0/170032257 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004009340 con 0x7f1014065b50 2024-09-10T18:36:57.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.239+0000 7f10198e0640 1 -- 172.21.15.100:0/170032257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101411ba50 con 0x7f1014065b50 2024-09-10T18:36:57.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.243+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff8005010 con 0x7f1014065b50 2024-09-10T18:36:57.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.243+0000 7f100b7fe640 1 -- 172.21.15.100:0/170032257 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f1004009730 con 0x7f1014065b50 2024-09-10T18:36:57.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.243+0000 7f100b7fe640 1 --2- 172.21.15.100:0/170032257 >> v2:172.21.15.100:6800/2155730476 conn(0x7f100006eec0 0x7f1000071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:57.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.243+0000 7f100b7fe640 1 -- 172.21.15.100:0/170032257 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f100408e2b0 con 0x7f1014065b50 2024-09-10T18:36:57.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.243+0000 7f100bfff640 1 --2- 172.21.15.100:0/170032257 >> v2:172.21.15.100:6800/2155730476 conn(0x7f100006eec0 0x7f1000071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:57.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.243+0000 7f100bfff640 1 --2- 172.21.15.100:0/170032257 >> v2:172.21.15.100:6800/2155730476 conn(0x7f100006eec0 0x7f1000071370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10140fd190 tx=0x7f0ffc00b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:57.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.251+0000 7f100b7fe640 1 -- 172.21.15.100:0/170032257 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f100405b150 con 0x7f1014065b50 2024-09-10T18:36:57.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.363+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f822004bf60 con 0x7f82200ff3f0 2024-09-10T18:36:57.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82177fe640 1 -- 172.21.15.100:0/2545336604 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8208019090 con 0x7f82200ff3f0 2024-09-10T18:36:57.369 INFO:teuthology.orchestra.run.smithi100.stdout:34359738488 2024-09-10T18:36:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 >> v2:172.21.15.100:6800/2155730476 conn(0x7f820c06edb0 msgr2=0x7f820c071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 --2- 172.21.15.100:0/2545336604 >> v2:172.21.15.100:6800/2155730476 conn(0x7f820c06edb0 0x7f820c071260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f82100115b0 tx=0x7f821000b040 comp rx=0 tx=0).stop 2024-09-10T18:36:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82200ff3f0 msgr2=0x7f8220114120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82200ff3f0 0x7f8220114120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f820800e990 tx=0x7f820800b040 comp rx=0 tx=0).stop 2024-09-10T18:36:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 shutdown_connections 2024-09-10T18:36:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 --2- 172.21.15.100:0/2545336604 >> v2:172.21.15.100:6800/2155730476 conn(0x7f820c06edb0 0x7f820c071260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8220104a60 0x7f822010ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f82200ff3f0 0x7f8220114120 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 --2- 172.21.15.100:0/2545336604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82200fea40 0x7f8220113be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 >> 172.21.15.100:0/2545336604 conn(0x7f82200fa5e0 msgr2=0x7f82200fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 shutdown_connections 2024-09-10T18:36:57.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.367+0000 7f82253ad640 1 -- 172.21.15.100:0/2545336604 wait complete. 2024-09-10T18:36:57.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.431+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0ff8005590 con 0x7f1014065b50 2024-09-10T18:36:57.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f100b7fe640 1 -- 172.21.15.100:0/170032257 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f100405aaf0 con 0x7f1014065b50 2024-09-10T18:36:57.436 INFO:teuthology.orchestra.run.smithi100.stdout:55834574959 2024-09-10T18:36:57.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 >> v2:172.21.15.100:6800/2155730476 conn(0x7f100006eec0 msgr2=0x7f1000071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 --2- 172.21.15.100:0/170032257 >> v2:172.21.15.100:6800/2155730476 conn(0x7f100006eec0 0x7f1000071370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10140fd190 tx=0x7f0ffc00b040 comp rx=0 tx=0).stop 2024-09-10T18:36:57.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 msgr2=0x7f10140fba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:57.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 0x7f10140fba60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f100400ebf0 tx=0x7f100400b040 comp rx=0 tx=0).stop 2024-09-10T18:36:57.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 shutdown_connections 2024-09-10T18:36:57.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 --2- 172.21.15.100:0/170032257 >> v2:172.21.15.100:6800/2155730476 conn(0x7f100006eec0 0x7f1000071370 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1014102850 0x7f10140fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1014066500 0x7f10140fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 --2- 172.21.15.100:0/170032257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1014065b50 0x7f10140fba60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:57.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 >> 172.21.15.100:0/170032257 conn(0x7f1014072470 msgr2=0x7f10140fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:57.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 shutdown_connections 2024-09-10T18:36:57.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:57.435+0000 7f10097fa640 1 -- 172.21.15.100:0/170032257 wait complete. 2024-09-10T18:36:58.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:57 smithi151 bash[22565]: audit 2024-09-10T18:36:57.369803+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? 172.21.15.100:0/2545336604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:36:58.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:57 smithi151 bash[22565]: audit 2024-09-10T18:36:57.437169+0000 mon.a (mon.0) 835 : audit [DBG] from='client.? 172.21.15.100:0/170032257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:36:58.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:57 smithi100 bash[31607]: audit 2024-09-10T18:36:57.369803+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? 172.21.15.100:0/2545336604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:36:58.187 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:57 smithi100 bash[31607]: audit 2024-09-10T18:36:57.437169+0000 mon.a (mon.0) 835 : audit [DBG] from='client.? 172.21.15.100:0/170032257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:36:58.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:57 smithi100 bash[18597]: audit 2024-09-10T18:36:57.369803+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? 172.21.15.100:0/2545336604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:36:58.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:57 smithi100 bash[18597]: audit 2024-09-10T18:36:57.437169+0000 mon.a (mon.0) 835 : audit [DBG] from='client.? 172.21.15.100:0/170032257' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- 172.21.15.100:0/376060023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 msgr2=0x7fd4dc107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/376060023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc107c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009cd0 tx=0x7fd4d4004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- 172.21.15.100:0/376060023 shutdown_connections 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/376060023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc107c80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/376060023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4dc0fb0b0 0x7fd4dc0fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/376060023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4dc100ef0 0x7fd4dc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- 172.21.15.100:0/376060023 >> 172.21.15.100:0/376060023 conn(0x7fd4dc071c00 msgr2=0x7fd4dc072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- 172.21.15.100:0/376060023 shutdown_connections 2024-09-10T18:36:58.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- 172.21.15.100:0/376060023 wait complete. 2024-09-10T18:36:58.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 Processor -- start 2024-09-10T18:36:58.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- start start 2024-09-10T18:36:58.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4dc0fb0b0 0x7fd4dc10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc10e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4dc100ef0 0x7fd4dc10e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc118fb0 con 0x7fd4dc0fba40 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc119120 con 0x7fd4dc100ef0 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4e1e05640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc1192e0 con 0x7fd4dc0fb0b0 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc10e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4db7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4dc0fb0b0 0x7fd4dc10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4daffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc10e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57748/0 (socket says 172.21.15.100:57748) 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4daffd640 1 -- 172.21.15.100:0/2716807543 learned_addr learned my addr 172.21.15.100:0/2716807543 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:36:58.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4dbfff640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4dc100ef0 0x7fd4dc10e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:58.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4dbfff640 1 -- 172.21.15.100:0/2716807543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4dc0fb0b0 msgr2=0x7fd4dc10ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:58.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4dbfff640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4dc0fb0b0 0x7fd4dc10ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4dbfff640 1 -- 172.21.15.100:0/2716807543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 msgr2=0x7fd4dc10e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:58.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4dbfff640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc10e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4dbfff640 1 -- 172.21.15.100:0/2716807543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0005040 con 0x7fd4dc100ef0 2024-09-10T18:36:58.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4daffd640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:36:58.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.299+0000 7fd4dbfff640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4dc100ef0 0x7fd4dc10e830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4d402a800 tx=0x7fd4d4004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:58.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.303+0000 7fd4c3fff640 1 -- 172.21.15.100:0/2716807543 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4038070 con 0x7fd4dc100ef0 2024-09-10T18:36:58.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.303+0000 7fd4c3fff640 1 -- 172.21.15.100:0/2716807543 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fd4d4033e10 con 0x7fd4dc100ef0 2024-09-10T18:36:58.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.303+0000 7fd4c3fff640 1 -- 172.21.15.100:0/2716807543 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d403c9e0 con 0x7fd4dc100ef0 2024-09-10T18:36:58.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.303+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d40098b0 con 0x7fd4dc100ef0 2024-09-10T18:36:58.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.303+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc119850 con 0x7fd4dc100ef0 2024-09-10T18:36:58.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.303+0000 7fd4c3fff640 1 -- 172.21.15.100:0/2716807543 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fd4d4030070 con 0x7fd4dc100ef0 2024-09-10T18:36:58.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.307+0000 7fd4c3fff640 1 --2- 172.21.15.100:0/2716807543 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd4c406eec0 0x7fd4c4071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:36:58.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.307+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc0fc5f0 con 0x7fd4dc100ef0 2024-09-10T18:36:58.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.307+0000 7fd4c3fff640 1 -- 172.21.15.100:0/2716807543 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7fd4d40aef10 con 0x7fd4dc100ef0 2024-09-10T18:36:58.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.307+0000 7fd4db7fe640 1 --2- 172.21.15.100:0/2716807543 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd4c406eec0 0x7fd4c4071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:36:58.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.307+0000 7fd4db7fe640 1 --2- 172.21.15.100:0/2716807543 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd4c406eec0 0x7fd4c4071370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009af0 tx=0x7fd4cc00b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:36:58.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.319+0000 7fd4c3fff640 1 -- 172.21.15.100:0/2716807543 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd4d407bdb0 con 0x7fd4dc100ef0 2024-09-10T18:36:58.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593778 got 184683593783 for osd.6 2024-09-10T18:36:58.335 DEBUG:teuthology.parallel:result is None 2024-09-10T18:36:58.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.503+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd4dc10a3c0 con 0x7fd4dc100ef0 2024-09-10T18:36:58.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.503+0000 7fd4c3fff640 1 -- 172.21.15.100:0/2716807543 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4d407b750 con 0x7fd4dc100ef0 2024-09-10T18:36:58.508 INFO:teuthology.orchestra.run.smithi100.stdout:158913790019 2024-09-10T18:36:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd4c406eec0 msgr2=0x7fd4c4071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/2716807543 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd4c406eec0 0x7fd4c4071370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009af0 tx=0x7fd4cc00b040 comp rx=0 tx=0).stop 2024-09-10T18:36:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4dc100ef0 msgr2=0x7fd4dc10e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:58.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4dc100ef0 0x7fd4dc10e830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4d402a800 tx=0x7fd4d4004d10 comp rx=0 tx=0).stop 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 shutdown_connections 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/2716807543 >> v2:172.21.15.100:6800/2155730476 conn(0x7fd4c406eec0 0x7fd4c4071370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4dc100ef0 0x7fd4dc10e830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4dc0fba40 0x7fd4dc10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 --2- 172.21.15.100:0/2716807543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4dc0fb0b0 0x7fd4dc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 >> 172.21.15.100:0/2716807543 conn(0x7fd4dc071c00 msgr2=0x7fd4dc0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 shutdown_connections 2024-09-10T18:36:58.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:36:58.507+0000 7fd4e1e05640 1 -- 172.21.15.100:0/2716807543 wait complete. 2024-09-10T18:36:59.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:58 smithi151 bash[22565]: cluster 2024-09-10T18:36:57.274098+0000 mgr.y (mgr.24512) 99 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:59.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:36:58 smithi151 bash[22565]: audit 2024-09-10T18:36:58.494541+0000 mon.b (mon.1) 35 : audit [DBG] from='client.? 172.21.15.100:0/2716807543' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:36:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:58 smithi100 bash[18597]: cluster 2024-09-10T18:36:57.274098+0000 mgr.y (mgr.24512) 99 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:36:58 smithi100 bash[18597]: audit 2024-09-10T18:36:58.494541+0000 mon.b (mon.1) 35 : audit [DBG] from='client.? 172.21.15.100:0/2716807543' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:36:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:59 smithi100 bash[31607]: cluster 2024-09-10T18:36:57.274098+0000 mgr.y (mgr.24512) 99 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:36:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:36:59 smithi100 bash[31607]: audit 2024-09-10T18:36:58.494541+0000 mon.b (mon.1) 35 : audit [DBG] from='client.? 172.21.15.100:0/2716807543' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:37:00.070 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574954 got 55834574959 for osd.1 2024-09-10T18:37:00.071 DEBUG:teuthology.parallel:result is None 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- 172.21.15.100:0/647661102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fb950 msgr2=0x7f9e400fbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 --2- 172.21.15.100:0/647661102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fb950 0x7f9e400fbdc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9e30004980 tx=0x7f9e30009da0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- 172.21.15.100:0/647661102 shutdown_connections 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 --2- 172.21.15.100:0/647661102 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e400fc300 0x7f9e40108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 --2- 172.21.15.100:0/647661102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fb950 0x7f9e400fbdc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 --2- 172.21.15.100:0/647661102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e40101760 0x7f9e40101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- 172.21.15.100:0/647661102 >> 172.21.15.100:0/647661102 conn(0x7f9e40072440 msgr2=0x7f9e40072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- 172.21.15.100:0/647661102 shutdown_connections 2024-09-10T18:37:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- 172.21.15.100:0/647661102 wait complete. 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 Processor -- start 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- start start 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e400fb950 0x7f9e40112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fc300 0x7f9e40113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e40101760 0x7f9e4010cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4010d530 con 0x7f9e400fc300 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4010d6a0 con 0x7f9e400fb950 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4727d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4010d860 con 0x7f9e40101760 2024-09-10T18:37:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e44818640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fc300 0x7f9e40113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4581a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e40101760 0x7f9e4010cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4581a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e40101760 0x7f9e4010cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:53644/0 (socket says 172.21.15.100:53644) 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e4581a640 1 -- 172.21.15.100:0/2445317380 learned_addr learned my addr 172.21.15.100:0/2445317380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e45019640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e400fb950 0x7f9e40112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e44818640 1 -- 172.21.15.100:0/2445317380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e40101760 msgr2=0x7f9e4010cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e44818640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e40101760 0x7f9e4010cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e44818640 1 -- 172.21.15.100:0/2445317380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e400fb950 msgr2=0x7f9e40112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e44818640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e400fb950 0x7f9e40112d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e44818640 1 -- 172.21.15.100:0/2445317380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30005040 con 0x7f9e400fc300 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e45019640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e400fb950 0x7f9e40112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.559+0000 7f9e44818640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fc300 0x7f9e40113290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9e30004980 tx=0x7f9e30009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e35ffb640 1 -- 172.21.15.100:0/2445317380 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3002ab50 con 0x7f9e400fc300 2024-09-10T18:37:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4010dae0 con 0x7f9e400fc300 2024-09-10T18:37:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e35ffb640 1 -- 172.21.15.100:0/2445317380 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f9e3002acb0 con 0x7f9e400fc300 2024-09-10T18:37:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e40119d40 con 0x7f9e400fc300 2024-09-10T18:37:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e35ffb640 1 -- 172.21.15.100:0/2445317380 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300323d0 con 0x7f9e400fc300 2024-09-10T18:37:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e35ffb640 1 -- 172.21.15.100:0/2445317380 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f9e30030020 con 0x7f9e400fc300 2024-09-10T18:37:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e400fce80 con 0x7f9e400fc300 2024-09-10T18:37:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e35ffb640 1 --2- 172.21.15.100:0/2445317380 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9e2c0732d0 0x7f9e2c075780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e35ffb640 1 -- 172.21.15.100:0/2445317380 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f9e300add40 con 0x7f9e400fc300 2024-09-10T18:37:00.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e45019640 1 --2- 172.21.15.100:0/2445317380 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9e2c0732d0 0x7f9e2c075780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.563+0000 7f9e45019640 1 --2- 172.21.15.100:0/2445317380 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9e2c0732d0 0x7f9e2c075780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009fd0 tx=0x7f9e3c009e50 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:00.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f9e35ffb640 1 -- 172.21.15.100:0/2445317380 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9e3007ac80 con 0x7f9e400fc300 2024-09-10T18:37:00.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- 172.21.15.100:0/3835802329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 msgr2=0x7f5554107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 --2- 172.21.15.100:0/3835802329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f5554107ba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5544004950 tx=0x7f5544009da0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- 172.21.15.100:0/3835802329 shutdown_connections 2024-09-10T18:37:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 --2- 172.21.15.100:0/3835802329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55541080e0 0x7f5554113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 --2- 172.21.15.100:0/3835802329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f5554107ba0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 --2- 172.21.15.100:0/3835802329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f555410d2a0 0x7f555410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- 172.21.15.100:0/3835802329 >> 172.21.15.100:0/3835802329 conn(0x7f5554067be0 msgr2=0x7f5554067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- 172.21.15.100:0/3835802329 shutdown_connections 2024-09-10T18:37:00.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- 172.21.15.100:0/3835802329 wait complete. 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 Processor -- start 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- start start 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f555411d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55541080e0 0x7f555411de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555410d2a0 0x7f5554117af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5554118140 con 0x7f5554107730 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55541182b0 con 0x7f55541080e0 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f555b6bb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5554118470 con 0x7f555410d2a0 2024-09-10T18:37:00.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f555411d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f555411d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48062/0 (socket says 172.21.15.100:48062) 2024-09-10T18:37:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 -- 172.21.15.100:0/840238617 learned_addr learned my addr 172.21.15.100:0/840238617 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:37:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559c58640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555410d2a0 0x7f5554117af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5558c56640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55541080e0 0x7f555411de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 -- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555410d2a0 msgr2=0x7f5554117af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555410d2a0 0x7f5554117af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 -- 172.21.15.100:0/840238617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55541080e0 msgr2=0x7f555411de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55541080e0 0x7f555411de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559457640 1 -- 172.21.15.100:0/840238617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5544005040 con 0x7f5554107730 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.571+0000 7f5559c58640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555410d2a0 0x7f5554117af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5559457640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f555411d940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5550002ad0 tx=0x7f5550004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5549ffb640 1 -- 172.21.15.100:0/840238617 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550007c30 con 0x7f5554107730 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5554118750 con 0x7f5554107730 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5549ffb640 1 -- 172.21.15.100:0/840238617 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f555000f040 con 0x7f5554107730 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5549ffb640 1 -- 172.21.15.100:0/840238617 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550009320 con 0x7f5554107730 2024-09-10T18:37:00.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5554124900 con 0x7f5554107730 2024-09-10T18:37:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5549ffb640 1 -- 172.21.15.100:0/840238617 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f55500096d0 con 0x7f5554107730 2024-09-10T18:37:00.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5554108990 con 0x7f5554107730 2024-09-10T18:37:00.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5549ffb640 1 --2- 172.21.15.100:0/840238617 >> v2:172.21.15.100:6800/2155730476 conn(0x7f554006eec0 0x7f5540071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:00.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5549ffb640 1 -- 172.21.15.100:0/840238617 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f555008e330 con 0x7f5554107730 2024-09-10T18:37:00.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.575+0000 7f5558c56640 1 --2- 172.21.15.100:0/840238617 >> v2:172.21.15.100:6800/2155730476 conn(0x7f554006eec0 0x7f5540071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:00.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.579+0000 7f5558c56640 1 --2- 172.21.15.100:0/840238617 >> v2:172.21.15.100:6800/2155730476 conn(0x7f554006eec0 0x7f5540071370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5544009040 tx=0x7f5544009da0 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:00.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.583+0000 7f5549ffb640 1 -- 172.21.15.100:0/840238617 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f555005b250 con 0x7f5554107730 2024-09-10T18:37:00.595 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738483 got 34359738488 for osd.0 2024-09-10T18:37:00.595 DEBUG:teuthology.parallel:result is None 2024-09-10T18:37:00.701 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790015 got 158913790019 for osd.5 2024-09-10T18:37:00.701 DEBUG:teuthology.parallel:result is None 2024-09-10T18:37:00.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.759+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e4010e3c0 con 0x7f9e400fc300 2024-09-10T18:37:00.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.759+0000 7f9e35ffb640 1 -- 172.21.15.100:0/2445317380 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e3007a620 con 0x7f9e400fc300 2024-09-10T18:37:00.763 INFO:teuthology.orchestra.run.smithi100.stdout:111669149785 2024-09-10T18:37:00.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9e2c0732d0 msgr2=0x7f9e2c075780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 --2- 172.21.15.100:0/2445317380 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9e2c0732d0 0x7f9e2c075780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009fd0 tx=0x7f9e3c009e50 comp rx=0 tx=0).stop 2024-09-10T18:37:00.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fc300 msgr2=0x7f9e40113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fc300 0x7f9e40113290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9e30004980 tx=0x7f9e30009da0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 shutdown_connections 2024-09-10T18:37:00.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 --2- 172.21.15.100:0/2445317380 >> v2:172.21.15.100:6800/2155730476 conn(0x7f9e2c0732d0 0x7f9e2c075780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e40101760 0x7f9e4010cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e400fc300 0x7f9e40113290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 --2- 172.21.15.100:0/2445317380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e400fb950 0x7f9e40112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 >> 172.21.15.100:0/2445317380 conn(0x7f9e40072440 msgr2=0x7f9e40065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:00.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 shutdown_connections 2024-09-10T18:37:00.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.763+0000 7f9e4727d640 1 -- 172.21.15.100:0/2445317380 wait complete. 2024-09-10T18:37:00.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.791+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f555404bf90 con 0x7f5554107730 2024-09-10T18:37:00.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.791+0000 7f5549ffb640 1 -- 172.21.15.100:0/840238617 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f555005abf0 con 0x7f5554107730 2024-09-10T18:37:00.793 INFO:teuthology.orchestra.run.smithi100.stdout:77309411428 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 >> v2:172.21.15.100:6800/2155730476 conn(0x7f554006eec0 msgr2=0x7f5540071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 --2- 172.21.15.100:0/840238617 >> v2:172.21.15.100:6800/2155730476 conn(0x7f554006eec0 0x7f5540071370 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5544009040 tx=0x7f5544009da0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 msgr2=0x7f555411d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f555411d940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5550002ad0 tx=0x7f5550004600 comp rx=0 tx=0).stop 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 shutdown_connections 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 --2- 172.21.15.100:0/840238617 >> v2:172.21.15.100:6800/2155730476 conn(0x7f554006eec0 0x7f5540071370 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f555410d2a0 0x7f5554117af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55541080e0 0x7f555411de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 --2- 172.21.15.100:0/840238617 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107730 0x7f555411d940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 >> 172.21.15.100:0/840238617 conn(0x7f5554067be0 msgr2=0x7f555410ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 shutdown_connections 2024-09-10T18:37:00.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:00.795+0000 7f555b6bb640 1 -- 172.21.15.100:0/840238617 wait complete. 2024-09-10T18:37:01.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:00 smithi100 bash[18597]: cluster 2024-09-10T18:36:59.274768+0000 mgr.y (mgr.24512) 100 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:01.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:00 smithi100 bash[18597]: audit 2024-09-10T18:37:00.764901+0000 mon.a (mon.0) 836 : audit [DBG] from='client.? 172.21.15.100:0/2445317380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:37:01.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:00 smithi100 bash[18597]: audit 2024-09-10T18:37:00.794985+0000 mon.a (mon.0) 837 : audit [DBG] from='client.? 172.21.15.100:0/840238617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:37:01.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:00 smithi100 bash[31607]: cluster 2024-09-10T18:36:59.274768+0000 mgr.y (mgr.24512) 100 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:01.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:00 smithi100 bash[31607]: audit 2024-09-10T18:37:00.764901+0000 mon.a (mon.0) 836 : audit [DBG] from='client.? 172.21.15.100:0/2445317380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:37:01.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:00 smithi100 bash[31607]: audit 2024-09-10T18:37:00.794985+0000 mon.a (mon.0) 837 : audit [DBG] from='client.? 172.21.15.100:0/840238617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:37:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:00 smithi151 bash[22565]: cluster 2024-09-10T18:36:59.274768+0000 mgr.y (mgr.24512) 100 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:00 smithi151 bash[22565]: audit 2024-09-10T18:37:00.764901+0000 mon.a (mon.0) 836 : audit [DBG] from='client.? 172.21.15.100:0/2445317380' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:37:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:00 smithi151 bash[22565]: audit 2024-09-10T18:37:00.794985+0000 mon.a (mon.0) 837 : audit [DBG] from='client.? 172.21.15.100:0/840238617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:37:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:37:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:01] "GET /metrics HTTP/1.1" 200 219674 "" "Prometheus/2.43.0" 2024-09-10T18:37:01.999 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149781 got 111669149785 for osd.3 2024-09-10T18:37:01.999 DEBUG:teuthology.parallel:result is None 2024-09-10T18:37:02.000 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411424 got 77309411428 for osd.2 2024-09-10T18:37:02.000 DEBUG:teuthology.parallel:result is None 2024-09-10T18:37:02.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:37:02.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:37:02.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:37:01 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:37:03.175 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:02 smithi151 bash[22565]: cluster 2024-09-10T18:37:01.276220+0000 mgr.y (mgr.24512) 101 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:03.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:02 smithi100 bash[31607]: cluster 2024-09-10T18:37:01.276220+0000 mgr.y (mgr.24512) 101 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:02 smithi100 bash[18597]: cluster 2024-09-10T18:37:01.276220+0000 mgr.y (mgr.24512) 101 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:03 smithi100 bash[18597]: audit 2024-09-10T18:37:01.969074+0000 mgr.y (mgr.24512) 102 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:04.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:03 smithi100 bash[31607]: audit 2024-09-10T18:37:01.969074+0000 mgr.y (mgr.24512) 102 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:04.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:03 smithi151 bash[22565]: audit 2024-09-10T18:37:01.969074+0000 mgr.y (mgr.24512) 102 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:04.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:37:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:37:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:04 smithi100 bash[18597]: cluster 2024-09-10T18:37:03.276933+0000 mgr.y (mgr.24512) 103 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:05.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:04 smithi100 bash[31607]: cluster 2024-09-10T18:37:03.276933+0000 mgr.y (mgr.24512) 103 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:05.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:04 smithi151 bash[22565]: cluster 2024-09-10T18:37:03.276933+0000 mgr.y (mgr.24512) 103 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:06.758 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:37:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:06 smithi100 bash[18597]: cluster 2024-09-10T18:37:05.278337+0000 mgr.y (mgr.24512) 104 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:07.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:06 smithi100 bash[31607]: cluster 2024-09-10T18:37:05.278337+0000 mgr.y (mgr.24512) 104 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:07.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:06 smithi151 bash[22565]: cluster 2024-09-10T18:37:05.278337+0000 mgr.y (mgr.24512) 104 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:08 smithi100 bash[18597]: cluster 2024-09-10T18:37:07.278989+0000 mgr.y (mgr.24512) 105 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:09.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:08 smithi100 bash[31607]: cluster 2024-09-10T18:37:07.278989+0000 mgr.y (mgr.24512) 105 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:09.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:08 smithi151 bash[22565]: cluster 2024-09-10T18:37:07.278989+0000 mgr.y (mgr.24512) 105 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:10.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- 172.21.15.100:0/3363363253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934065fd0 msgr2=0x7ff934066440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:10.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 --2- 172.21.15.100:0/3363363253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934065fd0 0x7ff934066440 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff928004950 tx=0x7ff928009da0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- 172.21.15.100:0/3363363253 shutdown_connections 2024-09-10T18:37:10.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 --2- 172.21.15.100:0/3363363253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff934066980 0x7ff934109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 --2- 172.21.15.100:0/3363363253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934065fd0 0x7ff934066440 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 --2- 172.21.15.100:0/3363363253 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934102850 0x7ff934102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- 172.21.15.100:0/3363363253 >> 172.21.15.100:0/3363363253 conn(0x7ff934072470 msgr2=0x7ff934072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:10.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- 172.21.15.100:0/3363363253 shutdown_connections 2024-09-10T18:37:10.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- 172.21.15.100:0/3363363253 wait complete. 2024-09-10T18:37:10.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 Processor -- start 2024-09-10T18:37:10.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- start start 2024-09-10T18:37:10.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934065fd0 0x7ff934112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934066980 0x7ff934113310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff934102850 0x7ff93410cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93410d5c0 con 0x7ff934066980 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93410d730 con 0x7ff934102850 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff93b587640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93410d8f0 con 0x7ff934065fd0 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff939323640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934065fd0 0x7ff934112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff938b22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934066980 0x7ff934113310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff939323640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934065fd0 0x7ff934112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:42110/0 (socket says 172.21.15.100:42110) 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff939323640 1 -- 172.21.15.100:0/1552433571 learned_addr learned my addr 172.21.15.100:0/1552433571 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff939b24640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff934102850 0x7ff93410cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff938b22640 1 -- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934065fd0 msgr2=0x7ff934112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:10.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff938b22640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934065fd0 0x7ff934112dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff938b22640 1 -- 172.21.15.100:0/1552433571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff934102850 msgr2=0x7ff93410cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff938b22640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff934102850 0x7ff93410cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff938b22640 1 -- 172.21.15.100:0/1552433571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c005040 con 0x7ff934066980 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff939323640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934065fd0 0x7ff934112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff939b24640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff934102850 0x7ff93410cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.123+0000 7ff938b22640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934066980 0x7ff934113310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff92802a480 tx=0x7ff928009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff931ffb640 1 -- 172.21.15.100:0/1552433571 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92802ab50 con 0x7ff934066980 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff931ffb640 1 -- 172.21.15.100:0/1552433571 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7ff92802acb0 con 0x7ff934066980 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff928005040 con 0x7ff934066980 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff931ffb640 1 -- 172.21.15.100:0/1552433571 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928033420 con 0x7ff934066980 2024-09-10T18:37:10.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff934119c20 con 0x7ff934066980 2024-09-10T18:37:10.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9340fdf70 con 0x7ff934066980 2024-09-10T18:37:10.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff931ffb640 1 -- 172.21.15.100:0/1552433571 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7ff928030020 con 0x7ff934066980 2024-09-10T18:37:10.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff931ffb640 1 --2- 172.21.15.100:0/1552433571 >> v2:172.21.15.100:6800/2155730476 conn(0x7ff91c06eec0 0x7ff91c071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:10.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff931ffb640 1 -- 172.21.15.100:0/1552433571 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7ff9280adf90 con 0x7ff934066980 2024-09-10T18:37:10.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.127+0000 7ff939323640 1 --2- 172.21.15.100:0/1552433571 >> v2:172.21.15.100:6800/2155730476 conn(0x7ff91c06eec0 0x7ff91c071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:10.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.131+0000 7ff939323640 1 --2- 172.21.15.100:0/1552433571 >> v2:172.21.15.100:6800/2155730476 conn(0x7ff91c06eec0 0x7ff91c071370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff92c004f40 tx=0x7ff92c00b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:10.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.135+0000 7ff931ffb640 1 -- 172.21.15.100:0/1552433571 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff92807aee0 con 0x7ff934066980 2024-09-10T18:37:10.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:09 smithi100 bash[31607]: audit 2024-09-10T18:37:09.317691+0000 mon.c (mon.2) 81 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:10.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:09 smithi100 bash[18597]: audit 2024-09-10T18:37:09.317691+0000 mon.c (mon.2) 81 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:10.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.307+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 --> v2:172.21.15.100:6800/2155730476 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff93404bfc0 con 0x7ff91c06eec0 2024-09-10T18:37:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.327+0000 7ff931ffb640 1 -- 172.21.15.100:0/1552433571 <== mgr.24512 v2:172.21.15.100:6800/2155730476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+382292 (secure 0 0 0) 0x7ff93404bfc0 con 0x7ff91c06eec0 2024-09-10T18:37:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:37:10.335 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-10T18:37:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 >> v2:172.21.15.100:6800/2155730476 conn(0x7ff91c06eec0 msgr2=0x7ff91c071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 --2- 172.21.15.100:0/1552433571 >> v2:172.21.15.100:6800/2155730476 conn(0x7ff91c06eec0 0x7ff91c071370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff92c004f40 tx=0x7ff92c00b040 comp rx=0 tx=0).stop 2024-09-10T18:37:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934066980 msgr2=0x7ff934113310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934066980 0x7ff934113310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff92802a480 tx=0x7ff928009da0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 shutdown_connections 2024-09-10T18:37:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 --2- 172.21.15.100:0/1552433571 >> v2:172.21.15.100:6800/2155730476 conn(0x7ff91c06eec0 0x7ff91c071370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff934102850 0x7ff93410cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934066980 0x7ff934113310 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 --2- 172.21.15.100:0/1552433571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934065fd0 0x7ff934112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 >> 172.21.15.100:0/1552433571 conn(0x7ff934072470 msgr2=0x7ff934100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 shutdown_connections 2024-09-10T18:37:10.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:10.331+0000 7ff93b587640 1 -- 172.21.15.100:0/1552433571 wait complete. 2024-09-10T18:37:10.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:09 smithi151 bash[22565]: audit 2024-09-10T18:37:09.317691+0000 mon.c (mon.2) 81 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:11.064 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-10T18:37:09.279294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1023,"num_read_kb":879,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":574,"ondisk_log_size":574,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385920,"kb_used_data":7072,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575296,"statfs":{"total":767960285184,"available":765517103104,"internally_reserved":0,"allocated":7241728,"data_stored":3884983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005484"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303531+0000","last_change":"2024-09-10T18:34:17.075989+0000","last_active":"2024-09-10T18:36:09.303531+0000","last_peered":"2024-09-10T18:36:09.303531+0000","last_clean":"2024-09-10T18:36:09.303531+0000","last_became_active":"2024-09-10T18:34:17.075423+0000","last_became_peered":"2024-09-10T18:34:17.075423+0000","last_unstale":"2024-09-10T18:36:09.303531+0000","last_undegraded":"2024-09-10T18:36:09.303531+0000","last_fullsized":"2024-09-10T18:36:09.303531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:07.113580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301433+0000","last_change":"2024-09-10T18:34:10.395850+0000","last_active":"2024-09-10T18:36:09.301433+0000","last_peered":"2024-09-10T18:36:09.301433+0000","last_clean":"2024-09-10T18:36:09.301433+0000","last_became_active":"2024-09-10T18:34:10.395374+0000","last_became_peered":"2024-09-10T18:34:10.395374+0000","last_unstale":"2024-09-10T18:36:09.301433+0000","last_undegraded":"2024-09-10T18:36:09.301433+0000","last_fullsized":"2024-09-10T18:36:09.301433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:39.666540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304665+0000","last_change":"2024-09-10T18:34:12.541967+0000","last_active":"2024-09-10T18:36:09.304665+0000","last_peered":"2024-09-10T18:36:09.304665+0000","last_clean":"2024-09-10T18:36:09.304665+0000","last_became_active":"2024-09-10T18:34:12.541697+0000","last_became_peered":"2024-09-10T18:34:12.541697+0000","last_unstale":"2024-09-10T18:36:09.304665+0000","last_undegraded":"2024-09-10T18:36:09.304665+0000","last_fullsized":"2024-09-10T18:36:09.304665+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:07.563690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303071+0000","last_change":"2024-09-10T18:34:14.757787+0000","last_active":"2024-09-10T18:36:09.303071+0000","last_peered":"2024-09-10T18:36:09.303071+0000","last_clean":"2024-09-10T18:36:09.303071+0000","last_became_active":"2024-09-10T18:34:14.757444+0000","last_became_peered":"2024-09-10T18:34:14.757444+0000","last_unstale":"2024-09-10T18:36:09.303071+0000","last_undegraded":"2024-09-10T18:36:09.303071+0000","last_fullsized":"2024-09-10T18:36:09.303071+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:55.055599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304583+0000","last_change":"2024-09-10T18:34:10.394278+0000","last_active":"2024-09-10T18:36:09.304583+0000","last_peered":"2024-09-10T18:36:09.304583+0000","last_clean":"2024-09-10T18:36:09.304583+0000","last_became_active":"2024-09-10T18:34:10.393723+0000","last_became_peered":"2024-09-10T18:34:10.393723+0000","last_unstale":"2024-09-10T18:36:09.304583+0000","last_undegraded":"2024-09-10T18:36:09.304583+0000","last_fullsized":"2024-09-10T18:36:09.304583+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:02.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298020+0000","last_change":"2024-09-10T18:34:12.543977+0000","last_active":"2024-09-10T18:36:09.298020+0000","last_peered":"2024-09-10T18:36:09.298020+0000","last_clean":"2024-09-10T18:36:09.298020+0000","last_became_active":"2024-09-10T18:34:12.543137+0000","last_became_peered":"2024-09-10T18:34:12.543137+0000","last_unstale":"2024-09-10T18:36:09.298020+0000","last_undegraded":"2024-09-10T18:36:09.298020+0000","last_fullsized":"2024-09-10T18:36:09.298020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:01.811278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283643+0000","last_change":"2024-09-10T18:34:14.754705+0000","last_active":"2024-09-10T18:36:09.283643+0000","last_peered":"2024-09-10T18:36:09.283643+0000","last_clean":"2024-09-10T18:36:09.283643+0000","last_became_active":"2024-09-10T18:34:14.754351+0000","last_became_peered":"2024-09-10T18:34:14.754351+0000","last_unstale":"2024-09-10T18:36:09.283643+0000","last_undegraded":"2024-09-10T18:36:09.283643+0000","last_fullsized":"2024-09-10T18:36:09.283643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:55.227711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283929+0000","last_change":"2024-09-10T18:34:16.774627+0000","last_active":"2024-09-10T18:36:09.283929+0000","last_peered":"2024-09-10T18:36:09.283929+0000","last_clean":"2024-09-10T18:36:09.283929+0000","last_became_active":"2024-09-10T18:34:16.774183+0000","last_became_peered":"2024-09-10T18:34:16.774183+0000","last_unstale":"2024-09-10T18:36:09.283929+0000","last_undegraded":"2024-09-10T18:36:09.283929+0000","last_fullsized":"2024-09-10T18:36:09.283929+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:54.090471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293199+0000","last_change":"2024-09-10T18:34:10.386014+0000","last_active":"2024-09-10T18:36:09.293199+0000","last_peered":"2024-09-10T18:36:09.293199+0000","last_clean":"2024-09-10T18:36:09.293199+0000","last_became_active":"2024-09-10T18:34:10.385139+0000","last_became_peered":"2024-09-10T18:34:10.385139+0000","last_unstale":"2024-09-10T18:36:09.293199+0000","last_undegraded":"2024-09-10T18:36:09.293199+0000","last_fullsized":"2024-09-10T18:36:09.293199+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:10.781879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294065+0000","last_change":"2024-09-10T18:34:12.412668+0000","last_active":"2024-09-10T18:36:09.294065+0000","last_peered":"2024-09-10T18:36:09.294065+0000","last_clean":"2024-09-10T18:36:09.294065+0000","last_became_active":"2024-09-10T18:34:12.412250+0000","last_became_peered":"2024-09-10T18:34:12.412250+0000","last_unstale":"2024-09-10T18:36:09.294065+0000","last_undegraded":"2024-09-10T18:36:09.294065+0000","last_fullsized":"2024-09-10T18:36:09.294065+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:29.440098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.296554+0000","last_change":"2024-09-10T18:36:10.296554+0000","last_active":"2024-09-10T18:36:10.296554+0000","last_peered":"2024-09-10T18:36:10.296554+0000","last_clean":"2024-09-10T18:36:10.296554+0000","last_became_active":"2024-09-10T18:36:10.295722+0000","last_became_peered":"2024-09-10T18:36:10.295722+0000","last_unstale":"2024-09-10T18:36:10.296554+0000","last_undegraded":"2024-09-10T18:36:10.296554+0000","last_fullsized":"2024-09-10T18:36:10.296554+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:43.381641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"60'1","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293881+0000","last_change":"2024-09-10T18:34:16.777489+0000","last_active":"2024-09-10T18:36:09.293881+0000","last_peered":"2024-09-10T18:36:09.293881+0000","last_clean":"2024-09-10T18:36:09.293881+0000","last_became_active":"2024-09-10T18:34:16.777161+0000","last_became_peered":"2024-09-10T18:34:16.777161+0000","last_unstale":"2024-09-10T18:36:09.293881+0000","last_undegraded":"2024-09-10T18:36:09.293881+0000","last_fullsized":"2024-09-10T18:36:09.293881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.516483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292998+0000","last_change":"2024-09-10T18:34:10.383837+0000","last_active":"2024-09-10T18:36:09.292998+0000","last_peered":"2024-09-10T18:36:09.292998+0000","last_clean":"2024-09-10T18:36:09.292998+0000","last_became_active":"2024-09-10T18:34:10.383543+0000","last_became_peered":"2024-09-10T18:34:10.383543+0000","last_unstale":"2024-09-10T18:36:09.292998+0000","last_undegraded":"2024-09-10T18:36:09.292998+0000","last_fullsized":"2024-09-10T18:36:09.292998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.480466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"60'12","reported_seq":44,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294816+0000","last_change":"2024-09-10T18:34:12.415075+0000","last_active":"2024-09-10T18:36:09.294816+0000","last_peered":"2024-09-10T18:36:09.294816+0000","last_clean":"2024-09-10T18:36:09.294816+0000","last_became_active":"2024-09-10T18:34:12.414803+0000","last_became_peered":"2024-09-10T18:34:12.414803+0000","last_unstale":"2024-09-10T18:36:09.294816+0000","last_undegraded":"2024-09-10T18:36:09.294816+0000","last_fullsized":"2024-09-10T18:36:09.294816+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:05.195754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284259+0000","last_change":"2024-09-10T18:34:14.755453+0000","last_active":"2024-09-10T18:36:09.284259+0000","last_peered":"2024-09-10T18:36:09.284259+0000","last_clean":"2024-09-10T18:36:09.284259+0000","last_became_active":"2024-09-10T18:34:14.755158+0000","last_became_peered":"2024-09-10T18:34:14.755158+0000","last_unstale":"2024-09-10T18:36:09.284259+0000","last_undegraded":"2024-09-10T18:36:09.284259+0000","last_fullsized":"2024-09-10T18:36:09.284259+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:37.262746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.296014+0000","last_change":"2024-09-10T18:34:16.791941+0000","last_active":"2024-09-10T18:36:09.296014+0000","last_peered":"2024-09-10T18:36:09.296014+0000","last_clean":"2024-09-10T18:36:09.296014+0000","last_became_active":"2024-09-10T18:34:16.791642+0000","last_became_peered":"2024-09-10T18:34:16.791642+0000","last_unstale":"2024-09-10T18:36:09.296014+0000","last_undegraded":"2024-09-10T18:36:09.296014+0000","last_fullsized":"2024-09-10T18:36:09.296014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:17.142556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"60'19","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235774+0000","last_change":"2024-09-10T18:34:12.408804+0000","last_active":"2024-09-10T18:35:09.235774+0000","last_peered":"2024-09-10T18:35:09.235774+0000","last_clean":"2024-09-10T18:35:09.235774+0000","last_became_active":"2024-09-10T18:34:12.408451+0000","last_became_peered":"2024-09-10T18:34:12.408451+0000","last_unstale":"2024-09-10T18:35:09.235774+0000","last_undegraded":"2024-09-10T18:35:09.235774+0000","last_fullsized":"2024-09-10T18:35:09.235774+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.810974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292788+0000","last_change":"2024-09-10T18:34:10.385622+0000","last_active":"2024-09-10T18:36:09.292788+0000","last_peered":"2024-09-10T18:36:09.292788+0000","last_clean":"2024-09-10T18:36:09.292788+0000","last_became_active":"2024-09-10T18:34:10.385331+0000","last_became_peered":"2024-09-10T18:34:10.385331+0000","last_unstale":"2024-09-10T18:36:09.292788+0000","last_undegraded":"2024-09-10T18:36:09.292788+0000","last_fullsized":"2024-09-10T18:36:09.292788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.539572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"60'11","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.833146+0000","last_change":"2024-09-10T18:34:14.757917+0000","last_active":"2024-09-10T18:36:58.833146+0000","last_peered":"2024-09-10T18:36:58.833146+0000","last_clean":"2024-09-10T18:36:58.833146+0000","last_became_active":"2024-09-10T18:34:14.757537+0000","last_became_peered":"2024-09-10T18:34:14.757537+0000","last_unstale":"2024-09-10T18:36:58.833146+0000","last_undegraded":"2024-09-10T18:36:58.833146+0000","last_fullsized":"2024-09-10T18:36:58.833146+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:37.472990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305299+0000","last_change":"2024-09-10T18:34:16.785700+0000","last_active":"2024-09-10T18:36:09.305299+0000","last_peered":"2024-09-10T18:36:09.305299+0000","last_clean":"2024-09-10T18:36:09.305299+0000","last_became_active":"2024-09-10T18:34:16.784170+0000","last_became_peered":"2024-09-10T18:34:16.784170+0000","last_unstale":"2024-09-10T18:36:09.305299+0000","last_undegraded":"2024-09-10T18:36:09.305299+0000","last_fullsized":"2024-09-10T18:36:09.305299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:45.303063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304283+0000","last_change":"2024-09-10T18:34:12.425361+0000","last_active":"2024-09-10T18:36:09.304283+0000","last_peered":"2024-09-10T18:36:09.304283+0000","last_clean":"2024-09-10T18:36:09.304283+0000","last_became_active":"2024-09-10T18:34:12.425037+0000","last_became_peered":"2024-09-10T18:34:12.425037+0000","last_unstale":"2024-09-10T18:36:09.304283+0000","last_undegraded":"2024-09-10T18:36:09.304283+0000","last_fullsized":"2024-09-10T18:36:09.304283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:26.724110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.299600+0000","last_change":"2024-09-10T18:34:10.394017+0000","last_active":"2024-09-10T18:36:09.299600+0000","last_peered":"2024-09-10T18:36:09.299600+0000","last_clean":"2024-09-10T18:36:09.299600+0000","last_became_active":"2024-09-10T18:34:10.393675+0000","last_became_peered":"2024-09-10T18:34:10.393675+0000","last_unstale":"2024-09-10T18:36:09.299600+0000","last_undegraded":"2024-09-10T18:36:09.299600+0000","last_fullsized":"2024-09-10T18:36:09.299600+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.977012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284490+0000","last_change":"2024-09-10T18:34:14.754706+0000","last_active":"2024-09-10T18:36:09.284490+0000","last_peered":"2024-09-10T18:36:09.284490+0000","last_clean":"2024-09-10T18:36:09.284490+0000","last_became_active":"2024-09-10T18:34:14.754412+0000","last_became_peered":"2024-09-10T18:34:14.754412+0000","last_unstale":"2024-09-10T18:36:09.284490+0000","last_undegraded":"2024-09-10T18:36:09.284490+0000","last_fullsized":"2024-09-10T18:36:09.284490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:14.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285226+0000","last_change":"2024-09-10T18:34:16.773881+0000","last_active":"2024-09-10T18:36:09.285226+0000","last_peered":"2024-09-10T18:36:09.285226+0000","last_clean":"2024-09-10T18:36:09.285226+0000","last_became_active":"2024-09-10T18:34:16.773546+0000","last_became_peered":"2024-09-10T18:34:16.773546+0000","last_unstale":"2024-09-10T18:36:09.285226+0000","last_undegraded":"2024-09-10T18:36:09.285226+0000","last_fullsized":"2024-09-10T18:36:09.285226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:01.758039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302501+0000","last_change":"2024-09-10T18:34:12.425588+0000","last_active":"2024-09-10T18:36:09.302501+0000","last_peered":"2024-09-10T18:36:09.302501+0000","last_clean":"2024-09-10T18:36:09.302501+0000","last_became_active":"2024-09-10T18:34:12.425168+0000","last_became_peered":"2024-09-10T18:34:12.425168+0000","last_unstale":"2024-09-10T18:36:09.302501+0000","last_undegraded":"2024-09-10T18:36:09.302501+0000","last_fullsized":"2024-09-10T18:36:09.302501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:22.016575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303716+0000","last_change":"2024-09-10T18:34:10.393121+0000","last_active":"2024-09-10T18:36:09.303716+0000","last_peered":"2024-09-10T18:36:09.303716+0000","last_clean":"2024-09-10T18:36:09.303716+0000","last_became_active":"2024-09-10T18:34:10.392805+0000","last_became_peered":"2024-09-10T18:34:10.392805+0000","last_unstale":"2024-09-10T18:36:09.303716+0000","last_undegraded":"2024-09-10T18:36:09.303716+0000","last_fullsized":"2024-09-10T18:36:09.303716+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:16.958863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304754+0000","last_change":"2024-09-10T18:34:14.766314+0000","last_active":"2024-09-10T18:36:09.304754+0000","last_peered":"2024-09-10T18:36:09.304754+0000","last_clean":"2024-09-10T18:36:09.304754+0000","last_became_active":"2024-09-10T18:34:14.765909+0000","last_became_peered":"2024-09-10T18:34:14.765909+0000","last_unstale":"2024-09-10T18:36:09.304754+0000","last_undegraded":"2024-09-10T18:36:09.304754+0000","last_fullsized":"2024-09-10T18:36:09.304754+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:32.963209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294414+0000","last_change":"2024-09-10T18:34:16.776881+0000","last_active":"2024-09-10T18:36:09.294414+0000","last_peered":"2024-09-10T18:36:09.294414+0000","last_clean":"2024-09-10T18:36:09.294414+0000","last_became_active":"2024-09-10T18:34:16.776413+0000","last_became_peered":"2024-09-10T18:34:16.776413+0000","last_unstale":"2024-09-10T18:36:09.294414+0000","last_undegraded":"2024-09-10T18:36:09.294414+0000","last_fullsized":"2024-09-10T18:36:09.294414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:59.503843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"60'12","reported_seq":44,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284487+0000","last_change":"2024-09-10T18:34:12.532970+0000","last_active":"2024-09-10T18:36:09.284487+0000","last_peered":"2024-09-10T18:36:09.284487+0000","last_clean":"2024-09-10T18:36:09.284487+0000","last_became_active":"2024-09-10T18:34:12.532083+0000","last_became_peered":"2024-09-10T18:34:12.532083+0000","last_unstale":"2024-09-10T18:36:09.284487+0000","last_undegraded":"2024-09-10T18:36:09.284487+0000","last_fullsized":"2024-09-10T18:36:09.284487+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:48.995415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291552+0000","last_change":"2024-09-10T18:34:10.386593+0000","last_active":"2024-09-10T18:36:09.291552+0000","last_peered":"2024-09-10T18:36:09.291552+0000","last_clean":"2024-09-10T18:36:09.291552+0000","last_became_active":"2024-09-10T18:34:10.386311+0000","last_became_peered":"2024-09-10T18:34:10.386311+0000","last_unstale":"2024-09-10T18:36:09.291552+0000","last_undegraded":"2024-09-10T18:36:09.291552+0000","last_fullsized":"2024-09-10T18:36:09.291552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:13.760929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.832617+0000","last_change":"2024-09-10T18:34:14.755395+0000","last_active":"2024-09-10T18:36:58.832617+0000","last_peered":"2024-09-10T18:36:58.832617+0000","last_clean":"2024-09-10T18:36:58.832617+0000","last_became_active":"2024-09-10T18:34:14.755096+0000","last_became_peered":"2024-09-10T18:34:14.755096+0000","last_unstale":"2024-09-10T18:36:58.832617+0000","last_undegraded":"2024-09-10T18:36:58.832617+0000","last_fullsized":"2024-09-10T18:36:58.832617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:24.068010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303027+0000","last_change":"2024-09-10T18:34:17.071114+0000","last_active":"2024-09-10T18:36:09.303027+0000","last_peered":"2024-09-10T18:36:09.303027+0000","last_clean":"2024-09-10T18:36:09.303027+0000","last_became_active":"2024-09-10T18:34:17.070207+0000","last_became_peered":"2024-09-10T18:34:17.070207+0000","last_unstale":"2024-09-10T18:36:09.303027+0000","last_undegraded":"2024-09-10T18:36:09.303027+0000","last_fullsized":"2024-09-10T18:36:09.303027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:53.923105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"60'12","reported_seq":39,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297432+0000","last_change":"2024-09-10T18:34:12.417008+0000","last_active":"2024-09-10T18:36:09.297432+0000","last_peered":"2024-09-10T18:36:09.297432+0000","last_clean":"2024-09-10T18:36:09.297432+0000","last_became_active":"2024-09-10T18:34:12.416463+0000","last_became_peered":"2024-09-10T18:34:12.416463+0000","last_unstale":"2024-09-10T18:36:09.297432+0000","last_undegraded":"2024-09-10T18:36:09.297432+0000","last_fullsized":"2024-09-10T18:36:09.297432+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:43.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.237443+0000","last_change":"2024-09-10T18:34:10.386812+0000","last_active":"2024-09-10T18:35:09.237443+0000","last_peered":"2024-09-10T18:35:09.237443+0000","last_clean":"2024-09-10T18:35:09.237443+0000","last_became_active":"2024-09-10T18:34:10.386530+0000","last_became_peered":"2024-09-10T18:34:10.386530+0000","last_unstale":"2024-09-10T18:35:09.237443+0000","last_undegraded":"2024-09-10T18:35:09.237443+0000","last_fullsized":"2024-09-10T18:35:09.237443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:23.383316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283765+0000","last_change":"2024-09-10T18:34:14.759668+0000","last_active":"2024-09-10T18:36:09.283765+0000","last_peered":"2024-09-10T18:36:09.283765+0000","last_clean":"2024-09-10T18:36:09.283765+0000","last_became_active":"2024-09-10T18:34:14.759403+0000","last_became_peered":"2024-09-10T18:34:14.759403+0000","last_unstale":"2024-09-10T18:36:09.283765+0000","last_undegraded":"2024-09-10T18:36:09.283765+0000","last_fullsized":"2024-09-10T18:36:09.283765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:10.185572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'5","reported_seq":175,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:04.150242+0000","last_change":"2024-09-10T18:34:26.912914+0000","last_active":"2024-09-10T18:37:04.150242+0000","last_peered":"2024-09-10T18:37:04.150242+0000","last_clean":"2024-09-10T18:37:04.150242+0000","last_became_active":"2024-09-10T18:34:15.776286+0000","last_became_peered":"2024-09-10T18:34:15.776286+0000","last_unstale":"2024-09-10T18:37:04.150242+0000","last_undegraded":"2024-09-10T18:37:04.150242+0000","last_fullsized":"2024-09-10T18:37:04.150242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_clean_scrub_stamp":"2024-09-10T18:34:14.742092+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:10.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014544790000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":149,"num_read_kb":144,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292347+0000","last_change":"2024-09-10T18:34:17.060699+0000","last_active":"2024-09-10T18:36:09.292347+0000","last_peered":"2024-09-10T18:36:09.292347+0000","last_clean":"2024-09-10T18:36:09.292347+0000","last_became_active":"2024-09-10T18:34:17.059974+0000","last_became_peered":"2024-09-10T18:34:17.059974+0000","last_unstale":"2024-09-10T18:36:09.292347+0000","last_undegraded":"2024-09-10T18:36:09.292347+0000","last_fullsized":"2024-09-10T18:36:09.292347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:22.982136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"60'13","reported_seq":48,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305015+0000","last_change":"2024-09-10T18:34:12.425358+0000","last_active":"2024-09-10T18:36:09.305015+0000","last_peered":"2024-09-10T18:36:09.305015+0000","last_clean":"2024-09-10T18:36:09.305015+0000","last_became_active":"2024-09-10T18:34:12.425014+0000","last_became_peered":"2024-09-10T18:34:12.425014+0000","last_unstale":"2024-09-10T18:36:09.305015+0000","last_undegraded":"2024-09-10T18:36:09.305015+0000","last_fullsized":"2024-09-10T18:36:09.305015+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:32.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"53'1","reported_seq":32,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304651+0000","last_change":"2024-09-10T18:34:10.388652+0000","last_active":"2024-09-10T18:36:09.304651+0000","last_peered":"2024-09-10T18:36:09.304651+0000","last_clean":"2024-09-10T18:36:09.304651+0000","last_became_active":"2024-09-10T18:34:10.388262+0000","last_became_peered":"2024-09-10T18:34:10.388262+0000","last_unstale":"2024-09-10T18:36:09.304651+0000","last_undegraded":"2024-09-10T18:36:09.304651+0000","last_fullsized":"2024-09-10T18:36:09.304651+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:55.976160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"60'11","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.846531+0000","last_change":"2024-09-10T18:34:14.757815+0000","last_active":"2024-09-10T18:36:58.846531+0000","last_peered":"2024-09-10T18:36:58.846531+0000","last_clean":"2024-09-10T18:36:58.846531+0000","last_became_active":"2024-09-10T18:34:14.757469+0000","last_became_peered":"2024-09-10T18:34:14.757469+0000","last_unstale":"2024-09-10T18:36:58.846531+0000","last_undegraded":"2024-09-10T18:36:58.846531+0000","last_fullsized":"2024-09-10T18:36:58.846531+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:11.124270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"61'1","reported_seq":24,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283719+0000","last_change":"2024-09-10T18:34:26.917522+0000","last_active":"2024-09-10T18:36:09.283719+0000","last_peered":"2024-09-10T18:36:09.283719+0000","last_clean":"2024-09-10T18:36:09.283719+0000","last_became_active":"2024-09-10T18:34:15.761086+0000","last_became_peered":"2024-09-10T18:34:15.761086+0000","last_unstale":"2024-09-10T18:36:09.283719+0000","last_undegraded":"2024-09-10T18:36:09.283719+0000","last_fullsized":"2024-09-10T18:36:09.283719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_clean_scrub_stamp":"2024-09-10T18:34:14.742092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:24.966817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086406500000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284148+0000","last_change":"2024-09-10T18:34:16.768373+0000","last_active":"2024-09-10T18:36:09.284148+0000","last_peered":"2024-09-10T18:36:09.284148+0000","last_clean":"2024-09-10T18:36:09.284148+0000","last_became_active":"2024-09-10T18:34:16.768067+0000","last_became_peered":"2024-09-10T18:34:16.768067+0000","last_unstale":"2024-09-10T18:36:09.284148+0000","last_undegraded":"2024-09-10T18:36:09.284148+0000","last_fullsized":"2024-09-10T18:36:09.284148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:25.947678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"60'30","reported_seq":106,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.846685+0000","last_change":"2024-09-10T18:34:12.544095+0000","last_active":"2024-09-10T18:36:58.846685+0000","last_peered":"2024-09-10T18:36:58.846685+0000","last_clean":"2024-09-10T18:36:58.846685+0000","last_became_active":"2024-09-10T18:34:12.543696+0000","last_became_peered":"2024-09-10T18:34:12.543696+0000","last_unstale":"2024-09-10T18:36:58.846685+0000","last_undegraded":"2024-09-10T18:36:58.846685+0000","last_fullsized":"2024-09-10T18:36:58.846685+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:59.615050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293198+0000","last_change":"2024-09-10T18:34:10.389037+0000","last_active":"2024-09-10T18:36:09.293198+0000","last_peered":"2024-09-10T18:36:09.293198+0000","last_clean":"2024-09-10T18:36:09.293198+0000","last_became_active":"2024-09-10T18:34:10.388734+0000","last_became_peered":"2024-09-10T18:34:10.388734+0000","last_unstale":"2024-09-10T18:36:09.293198+0000","last_undegraded":"2024-09-10T18:36:09.293198+0000","last_fullsized":"2024-09-10T18:36:09.293198+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:40.596707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"60'11","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.846496+0000","last_change":"2024-09-10T18:34:14.765846+0000","last_active":"2024-09-10T18:36:58.846496+0000","last_peered":"2024-09-10T18:36:58.846496+0000","last_clean":"2024-09-10T18:36:58.846496+0000","last_became_active":"2024-09-10T18:34:14.765312+0000","last_became_peered":"2024-09-10T18:34:14.765312+0000","last_unstale":"2024-09-10T18:36:58.846496+0000","last_undegraded":"2024-09-10T18:36:58.846496+0000","last_fullsized":"2024-09-10T18:36:58.846496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:20.012037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"63'2","reported_seq":24,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235156+0000","last_change":"2024-09-10T18:34:26.907339+0000","last_active":"2024-09-10T18:35:09.235156+0000","last_peered":"2024-09-10T18:35:09.235156+0000","last_clean":"2024-09-10T18:35:09.235156+0000","last_became_active":"2024-09-10T18:34:15.760794+0000","last_became_peered":"2024-09-10T18:34:15.760794+0000","last_unstale":"2024-09-10T18:35:09.235156+0000","last_undegraded":"2024-09-10T18:35:09.235156+0000","last_fullsized":"2024-09-10T18:35:09.235156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_clean_scrub_stamp":"2024-09-10T18:34:14.742092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:58.281474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001795804,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303435+0000","last_change":"2024-09-10T18:34:17.071309+0000","last_active":"2024-09-10T18:36:09.303435+0000","last_peered":"2024-09-10T18:36:09.303435+0000","last_clean":"2024-09-10T18:36:09.303435+0000","last_became_active":"2024-09-10T18:34:17.070392+0000","last_became_peered":"2024-09-10T18:34:17.070392+0000","last_unstale":"2024-09-10T18:36:09.303435+0000","last_undegraded":"2024-09-10T18:36:09.303435+0000","last_fullsized":"2024-09-10T18:36:09.303435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:50.193185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"60'14","reported_seq":43,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294796+0000","last_change":"2024-09-10T18:34:12.528911+0000","last_active":"2024-09-10T18:36:09.294796+0000","last_peered":"2024-09-10T18:36:09.294796+0000","last_clean":"2024-09-10T18:36:09.294796+0000","last_became_active":"2024-09-10T18:34:12.528095+0000","last_became_peered":"2024-09-10T18:34:12.528095+0000","last_unstale":"2024-09-10T18:36:09.294796+0000","last_undegraded":"2024-09-10T18:36:09.294796+0000","last_fullsized":"2024-09-10T18:36:09.294796+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.917283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.311047+0000","last_change":"2024-09-10T18:36:10.311047+0000","last_active":"2024-09-10T18:36:10.311047+0000","last_peered":"2024-09-10T18:36:10.311047+0000","last_clean":"2024-09-10T18:36:10.311047+0000","last_became_active":"2024-09-10T18:36:10.310735+0000","last_became_peered":"2024-09-10T18:36:10.310735+0000","last_unstale":"2024-09-10T18:36:10.311047+0000","last_undegraded":"2024-09-10T18:36:10.311047+0000","last_fullsized":"2024-09-10T18:36:10.311047+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:27.207694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298932+0000","last_change":"2024-09-10T18:34:14.770112+0000","last_active":"2024-09-10T18:36:09.298932+0000","last_peered":"2024-09-10T18:36:09.298932+0000","last_clean":"2024-09-10T18:36:09.298932+0000","last_became_active":"2024-09-10T18:34:14.769800+0000","last_became_peered":"2024-09-10T18:34:14.769800+0000","last_unstale":"2024-09-10T18:36:09.298932+0000","last_undegraded":"2024-09-10T18:36:09.298932+0000","last_fullsized":"2024-09-10T18:36:09.298932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:28.581637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.296280+0000","last_change":"2024-09-10T18:34:16.788769+0000","last_active":"2024-09-10T18:36:09.296280+0000","last_peered":"2024-09-10T18:36:09.296280+0000","last_clean":"2024-09-10T18:36:09.296280+0000","last_became_active":"2024-09-10T18:34:16.788476+0000","last_became_peered":"2024-09-10T18:34:16.788476+0000","last_unstale":"2024-09-10T18:36:09.296280+0000","last_undegraded":"2024-09-10T18:36:09.296280+0000","last_fullsized":"2024-09-10T18:36:09.296280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:28.124907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"60'19","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.286040+0000","last_change":"2024-09-10T18:34:12.410505+0000","last_active":"2024-09-10T18:36:09.286040+0000","last_peered":"2024-09-10T18:36:09.286040+0000","last_clean":"2024-09-10T18:36:09.286040+0000","last_became_active":"2024-09-10T18:34:12.410091+0000","last_became_peered":"2024-09-10T18:34:12.410091+0000","last_unstale":"2024-09-10T18:36:09.286040+0000","last_undegraded":"2024-09-10T18:36:09.286040+0000","last_fullsized":"2024-09-10T18:36:09.286040+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:27.929172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294709+0000","last_change":"2024-09-10T18:34:10.381107+0000","last_active":"2024-09-10T18:36:09.294709+0000","last_peered":"2024-09-10T18:36:09.294709+0000","last_clean":"2024-09-10T18:36:09.294709+0000","last_became_active":"2024-09-10T18:34:10.380610+0000","last_became_peered":"2024-09-10T18:34:10.380610+0000","last_unstale":"2024-09-10T18:36:09.294709+0000","last_undegraded":"2024-09-10T18:36:09.294709+0000","last_fullsized":"2024-09-10T18:36:09.294709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:46.895780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.300231+0000","last_change":"2024-09-10T18:34:14.762478+0000","last_active":"2024-09-10T18:36:09.300231+0000","last_peered":"2024-09-10T18:36:09.300231+0000","last_clean":"2024-09-10T18:36:09.300231+0000","last_became_active":"2024-09-10T18:34:14.761726+0000","last_became_peered":"2024-09-10T18:34:14.761726+0000","last_unstale":"2024-09-10T18:36:09.300231+0000","last_undegraded":"2024-09-10T18:36:09.300231+0000","last_fullsized":"2024-09-10T18:36:09.300231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:44.927582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303953+0000","last_change":"2024-09-10T18:34:16.786378+0000","last_active":"2024-09-10T18:36:09.303953+0000","last_peered":"2024-09-10T18:36:09.303953+0000","last_clean":"2024-09-10T18:36:09.303953+0000","last_became_active":"2024-09-10T18:34:16.786077+0000","last_became_peered":"2024-09-10T18:34:16.786077+0000","last_unstale":"2024-09-10T18:36:09.303953+0000","last_undegraded":"2024-09-10T18:36:09.303953+0000","last_fullsized":"2024-09-10T18:36:09.303953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:24.172589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"60'18","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.295939+0000","last_change":"2024-09-10T18:34:12.540632+0000","last_active":"2024-09-10T18:36:09.295939+0000","last_peered":"2024-09-10T18:36:09.295939+0000","last_clean":"2024-09-10T18:36:09.295939+0000","last_became_active":"2024-09-10T18:34:12.540348+0000","last_became_peered":"2024-09-10T18:34:12.540348+0000","last_unstale":"2024-09-10T18:36:09.295939+0000","last_undegraded":"2024-09-10T18:36:09.295939+0000","last_fullsized":"2024-09-10T18:36:09.295939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:51.309938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305462+0000","last_change":"2024-09-10T18:34:10.397178+0000","last_active":"2024-09-10T18:36:09.305462+0000","last_peered":"2024-09-10T18:36:09.305462+0000","last_clean":"2024-09-10T18:36:09.305462+0000","last_became_active":"2024-09-10T18:34:10.396865+0000","last_became_peered":"2024-09-10T18:34:10.396865+0000","last_unstale":"2024-09-10T18:36:09.305462+0000","last_undegraded":"2024-09-10T18:36:09.305462+0000","last_fullsized":"2024-09-10T18:36:09.305462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:42.452022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.663025+0000","last_change":"2024-09-10T18:36:10.663025+0000","last_active":"2024-09-10T18:36:10.663025+0000","last_peered":"2024-09-10T18:36:10.663025+0000","last_clean":"2024-09-10T18:36:10.663025+0000","last_became_active":"2024-09-10T18:36:10.662169+0000","last_became_peered":"2024-09-10T18:36:10.662169+0000","last_unstale":"2024-09-10T18:36:10.663025+0000","last_undegraded":"2024-09-10T18:36:10.663025+0000","last_fullsized":"2024-09-10T18:36:10.663025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:43.803320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292694+0000","last_change":"2024-09-10T18:34:17.062882+0000","last_active":"2024-09-10T18:36:09.292694+0000","last_peered":"2024-09-10T18:36:09.292694+0000","last_clean":"2024-09-10T18:36:09.292694+0000","last_became_active":"2024-09-10T18:34:17.062544+0000","last_became_peered":"2024-09-10T18:34:17.062544+0000","last_unstale":"2024-09-10T18:36:09.292694+0000","last_undegraded":"2024-09-10T18:36:09.292694+0000","last_fullsized":"2024-09-10T18:36:09.292694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:12.227256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"60'14","reported_seq":42,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298912+0000","last_change":"2024-09-10T18:34:12.419858+0000","last_active":"2024-09-10T18:36:09.298912+0000","last_peered":"2024-09-10T18:36:09.298912+0000","last_clean":"2024-09-10T18:36:09.298912+0000","last_became_active":"2024-09-10T18:34:12.419525+0000","last_became_peered":"2024-09-10T18:34:12.419525+0000","last_unstale":"2024-09-10T18:36:09.298912+0000","last_undegraded":"2024-09-10T18:36:09.298912+0000","last_fullsized":"2024-09-10T18:36:09.298912+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:56.699819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291680+0000","last_change":"2024-09-10T18:34:10.386886+0000","last_active":"2024-09-10T18:36:09.291680+0000","last_peered":"2024-09-10T18:36:09.291680+0000","last_clean":"2024-09-10T18:36:09.291680+0000","last_became_active":"2024-09-10T18:34:10.386597+0000","last_became_peered":"2024-09-10T18:34:10.386597+0000","last_unstale":"2024-09-10T18:36:09.291680+0000","last_undegraded":"2024-09-10T18:36:09.291680+0000","last_fullsized":"2024-09-10T18:36:09.291680+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:05.511409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297564+0000","last_change":"2024-09-10T18:34:14.760467+0000","last_active":"2024-09-10T18:36:09.297564+0000","last_peered":"2024-09-10T18:36:09.297564+0000","last_clean":"2024-09-10T18:36:09.297564+0000","last_became_active":"2024-09-10T18:34:14.760125+0000","last_became_peered":"2024-09-10T18:34:14.760125+0000","last_unstale":"2024-09-10T18:36:09.297564+0000","last_undegraded":"2024-09-10T18:36:09.297564+0000","last_fullsized":"2024-09-10T18:36:09.297564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:49.349310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.300108+0000","last_change":"2024-09-10T18:34:16.788735+0000","last_active":"2024-09-10T18:36:09.300108+0000","last_peered":"2024-09-10T18:36:09.300108+0000","last_clean":"2024-09-10T18:36:09.300108+0000","last_became_active":"2024-09-10T18:34:16.788438+0000","last_became_peered":"2024-09-10T18:34:16.788438+0000","last_unstale":"2024-09-10T18:36:09.300108+0000","last_undegraded":"2024-09-10T18:36:09.300108+0000","last_fullsized":"2024-09-10T18:36:09.300108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:18.081970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302731+0000","last_change":"2024-09-10T18:34:12.426576+0000","last_active":"2024-09-10T18:36:09.302731+0000","last_peered":"2024-09-10T18:36:09.302731+0000","last_clean":"2024-09-10T18:36:09.302731+0000","last_became_active":"2024-09-10T18:34:12.426080+0000","last_became_peered":"2024-09-10T18:34:12.426080+0000","last_unstale":"2024-09-10T18:36:09.302731+0000","last_undegraded":"2024-09-10T18:36:09.302731+0000","last_fullsized":"2024-09-10T18:36:09.302731+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:47.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294381+0000","last_change":"2024-09-10T18:34:10.378622+0000","last_active":"2024-09-10T18:36:09.294381+0000","last_peered":"2024-09-10T18:36:09.294381+0000","last_clean":"2024-09-10T18:36:09.294381+0000","last_became_active":"2024-09-10T18:34:10.378395+0000","last_became_peered":"2024-09-10T18:34:10.378395+0000","last_unstale":"2024-09-10T18:36:09.294381+0000","last_undegraded":"2024-09-10T18:36:09.294381+0000","last_fullsized":"2024-09-10T18:36:09.294381+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:39.264870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"65'79","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291877+0000","last_change":"2024-09-10T18:33:28.631139+0000","last_active":"2024-09-10T18:36:09.291877+0000","last_peered":"2024-09-10T18:36:09.291877+0000","last_clean":"2024-09-10T18:36:09.291877+0000","last_became_active":"2024-09-10T18:33:28.620160+0000","last_became_peered":"2024-09-10T18:33:28.620160+0000","last_unstale":"2024-09-10T18:36:09.291877+0000","last_undegraded":"2024-09-10T18:36:09.291877+0000","last_fullsized":"2024-09-10T18:36:09.291877+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:28:49.953203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:28:49.953203+0000","last_clean_scrub_stamp":"2024-09-10T18:28:49.953203+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:01.528284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.236361+0000","last_change":"2024-09-10T18:34:14.760355+0000","last_active":"2024-09-10T18:35:09.236361+0000","last_peered":"2024-09-10T18:35:09.236361+0000","last_clean":"2024-09-10T18:35:09.236361+0000","last_became_active":"2024-09-10T18:34:14.760023+0000","last_became_peered":"2024-09-10T18:34:14.760023+0000","last_unstale":"2024-09-10T18:35:09.236361+0000","last_undegraded":"2024-09-10T18:35:09.236361+0000","last_fullsized":"2024-09-10T18:35:09.236361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:54.765145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294691+0000","last_change":"2024-09-10T18:34:16.775535+0000","last_active":"2024-09-10T18:36:09.294691+0000","last_peered":"2024-09-10T18:36:09.294691+0000","last_clean":"2024-09-10T18:36:09.294691+0000","last_became_active":"2024-09-10T18:34:16.775183+0000","last_became_peered":"2024-09-10T18:34:16.775183+0000","last_unstale":"2024-09-10T18:36:09.294691+0000","last_undegraded":"2024-09-10T18:36:09.294691+0000","last_fullsized":"2024-09-10T18:36:09.294691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:25.153121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"60'17","reported_seq":49,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293071+0000","last_change":"2024-09-10T18:34:12.412677+0000","last_active":"2024-09-10T18:36:09.293071+0000","last_peered":"2024-09-10T18:36:09.293071+0000","last_clean":"2024-09-10T18:36:09.293071+0000","last_became_active":"2024-09-10T18:34:12.412384+0000","last_became_peered":"2024-09-10T18:34:12.412384+0000","last_unstale":"2024-09-10T18:36:09.293071+0000","last_undegraded":"2024-09-10T18:36:09.293071+0000","last_fullsized":"2024-09-10T18:36:09.293071+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:41.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303991+0000","last_change":"2024-09-10T18:34:10.390836+0000","last_active":"2024-09-10T18:36:09.303991+0000","last_peered":"2024-09-10T18:36:09.303991+0000","last_clean":"2024-09-10T18:36:09.303991+0000","last_became_active":"2024-09-10T18:34:10.390162+0000","last_became_peered":"2024-09-10T18:34:10.390162+0000","last_unstale":"2024-09-10T18:36:09.303991+0000","last_undegraded":"2024-09-10T18:36:09.303991+0000","last_fullsized":"2024-09-10T18:36:09.303991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:38.705903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235346+0000","last_change":"2024-09-10T18:34:14.753040+0000","last_active":"2024-09-10T18:35:09.235346+0000","last_peered":"2024-09-10T18:35:09.235346+0000","last_clean":"2024-09-10T18:35:09.235346+0000","last_became_active":"2024-09-10T18:34:14.752804+0000","last_became_peered":"2024-09-10T18:34:14.752804+0000","last_unstale":"2024-09-10T18:35:09.235346+0000","last_undegraded":"2024-09-10T18:35:09.235346+0000","last_fullsized":"2024-09-10T18:35:09.235346+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.654922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291774+0000","last_change":"2024-09-10T18:34:16.779778+0000","last_active":"2024-09-10T18:36:09.291774+0000","last_peered":"2024-09-10T18:36:09.291774+0000","last_clean":"2024-09-10T18:36:09.291774+0000","last_became_active":"2024-09-10T18:34:16.779441+0000","last_became_peered":"2024-09-10T18:34:16.779441+0000","last_unstale":"2024-09-10T18:36:09.291774+0000","last_undegraded":"2024-09-10T18:36:09.291774+0000","last_fullsized":"2024-09-10T18:36:09.291774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:51.472906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294242+0000","last_change":"2024-09-10T18:34:12.414377+0000","last_active":"2024-09-10T18:36:09.294242+0000","last_peered":"2024-09-10T18:36:09.294242+0000","last_clean":"2024-09-10T18:36:09.294242+0000","last_became_active":"2024-09-10T18:34:12.413793+0000","last_became_peered":"2024-09-10T18:34:12.413793+0000","last_unstale":"2024-09-10T18:36:09.294242+0000","last_undegraded":"2024-09-10T18:36:09.294242+0000","last_fullsized":"2024-09-10T18:36:09.294242+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:21.235880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301544+0000","last_change":"2024-09-10T18:34:10.384603+0000","last_active":"2024-09-10T18:36:09.301544+0000","last_peered":"2024-09-10T18:36:09.301544+0000","last_clean":"2024-09-10T18:36:09.301544+0000","last_became_active":"2024-09-10T18:34:10.384238+0000","last_became_peered":"2024-09-10T18:34:10.384238+0000","last_unstale":"2024-09-10T18:36:09.301544+0000","last_undegraded":"2024-09-10T18:36:09.301544+0000","last_fullsized":"2024-09-10T18:36:09.301544+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:37.702399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297804+0000","last_change":"2024-09-10T18:34:14.773539+0000","last_active":"2024-09-10T18:36:09.297804+0000","last_peered":"2024-09-10T18:36:09.297804+0000","last_clean":"2024-09-10T18:36:09.297804+0000","last_became_active":"2024-09-10T18:34:14.773211+0000","last_became_peered":"2024-09-10T18:34:14.773211+0000","last_unstale":"2024-09-10T18:36:09.297804+0000","last_undegraded":"2024-09-10T18:36:09.297804+0000","last_fullsized":"2024-09-10T18:36:09.297804+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:59.702492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298775+0000","last_change":"2024-09-10T18:34:16.790464+0000","last_active":"2024-09-10T18:36:09.298775+0000","last_peered":"2024-09-10T18:36:09.298775+0000","last_clean":"2024-09-10T18:36:09.298775+0000","last_became_active":"2024-09-10T18:34:16.790053+0000","last_became_peered":"2024-09-10T18:34:16.790053+0000","last_unstale":"2024-09-10T18:36:09.298775+0000","last_undegraded":"2024-09-10T18:36:09.298775+0000","last_fullsized":"2024-09-10T18:36:09.298775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:39.152799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292948+0000","last_change":"2024-09-10T18:34:12.415136+0000","last_active":"2024-09-10T18:36:09.292948+0000","last_peered":"2024-09-10T18:36:09.292948+0000","last_clean":"2024-09-10T18:36:09.292948+0000","last_became_active":"2024-09-10T18:34:12.414635+0000","last_became_peered":"2024-09-10T18:34:12.414635+0000","last_unstale":"2024-09-10T18:36:09.292948+0000","last_undegraded":"2024-09-10T18:36:09.292948+0000","last_fullsized":"2024-09-10T18:36:09.292948+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:57.182105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302353+0000","last_change":"2024-09-10T18:34:10.393336+0000","last_active":"2024-09-10T18:36:09.302353+0000","last_peered":"2024-09-10T18:36:09.302353+0000","last_clean":"2024-09-10T18:36:09.302353+0000","last_became_active":"2024-09-10T18:34:10.392912+0000","last_became_peered":"2024-09-10T18:34:10.392912+0000","last_unstale":"2024-09-10T18:36:09.302353+0000","last_undegraded":"2024-09-10T18:36:09.302353+0000","last_fullsized":"2024-09-10T18:36:09.302353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:51.401558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293676+0000","last_change":"2024-09-10T18:34:14.752413+0000","last_active":"2024-09-10T18:36:09.293676+0000","last_peered":"2024-09-10T18:36:09.293676+0000","last_clean":"2024-09-10T18:36:09.293676+0000","last_became_active":"2024-09-10T18:34:14.751793+0000","last_became_peered":"2024-09-10T18:34:14.751793+0000","last_unstale":"2024-09-10T18:36:09.293676+0000","last_undegraded":"2024-09-10T18:36:09.293676+0000","last_fullsized":"2024-09-10T18:36:09.293676+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:18.884416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294234+0000","last_change":"2024-09-10T18:34:17.057325+0000","last_active":"2024-09-10T18:36:09.294234+0000","last_peered":"2024-09-10T18:36:09.294234+0000","last_clean":"2024-09-10T18:36:09.294234+0000","last_became_active":"2024-09-10T18:34:17.056572+0000","last_became_peered":"2024-09-10T18:34:17.056572+0000","last_unstale":"2024-09-10T18:36:09.294234+0000","last_undegraded":"2024-09-10T18:36:09.294234+0000","last_fullsized":"2024-09-10T18:36:09.294234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:43.185973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293442+0000","last_change":"2024-09-10T18:34:12.415162+0000","last_active":"2024-09-10T18:36:09.293442+0000","last_peered":"2024-09-10T18:36:09.293442+0000","last_clean":"2024-09-10T18:36:09.293442+0000","last_became_active":"2024-09-10T18:34:12.414822+0000","last_became_peered":"2024-09-10T18:34:12.414822+0000","last_unstale":"2024-09-10T18:36:09.293442+0000","last_undegraded":"2024-09-10T18:36:09.293442+0000","last_fullsized":"2024-09-10T18:36:09.293442+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:56.490961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285490+0000","last_change":"2024-09-10T18:34:10.384963+0000","last_active":"2024-09-10T18:36:09.285490+0000","last_peered":"2024-09-10T18:36:09.285490+0000","last_clean":"2024-09-10T18:36:09.285490+0000","last_became_active":"2024-09-10T18:34:10.384662+0000","last_became_peered":"2024-09-10T18:34:10.384662+0000","last_unstale":"2024-09-10T18:36:09.285490+0000","last_undegraded":"2024-09-10T18:36:09.285490+0000","last_fullsized":"2024-09-10T18:36:09.285490+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:47.505769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"60'11","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.833176+0000","last_change":"2024-09-10T18:34:14.758937+0000","last_active":"2024-09-10T18:36:58.833176+0000","last_peered":"2024-09-10T18:36:58.833176+0000","last_clean":"2024-09-10T18:36:58.833176+0000","last_became_active":"2024-09-10T18:34:14.758572+0000","last_became_peered":"2024-09-10T18:34:14.758572+0000","last_unstale":"2024-09-10T18:36:58.833176+0000","last_undegraded":"2024-09-10T18:36:58.833176+0000","last_fullsized":"2024-09-10T18:36:58.833176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:34.905779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304306+0000","last_change":"2024-09-10T18:34:16.786561+0000","last_active":"2024-09-10T18:36:09.304306+0000","last_peered":"2024-09-10T18:36:09.304306+0000","last_clean":"2024-09-10T18:36:09.304306+0000","last_became_active":"2024-09-10T18:34:16.786264+0000","last_became_peered":"2024-09-10T18:34:16.786264+0000","last_unstale":"2024-09-10T18:36:09.304306+0000","last_undegraded":"2024-09-10T18:36:09.304306+0000","last_fullsized":"2024-09-10T18:36:09.304306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:41.116785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292189+0000","last_change":"2024-09-10T18:34:12.414409+0000","last_active":"2024-09-10T18:36:09.292189+0000","last_peered":"2024-09-10T18:36:09.292189+0000","last_clean":"2024-09-10T18:36:09.292189+0000","last_became_active":"2024-09-10T18:34:12.413996+0000","last_became_peered":"2024-09-10T18:34:12.413996+0000","last_unstale":"2024-09-10T18:36:09.292189+0000","last_undegraded":"2024-09-10T18:36:09.292189+0000","last_fullsized":"2024-09-10T18:36:09.292189+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:10.686059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304256+0000","last_change":"2024-09-10T18:34:10.390412+0000","last_active":"2024-09-10T18:36:09.304256+0000","last_peered":"2024-09-10T18:36:09.304256+0000","last_clean":"2024-09-10T18:36:09.304256+0000","last_became_active":"2024-09-10T18:34:10.390061+0000","last_became_peered":"2024-09-10T18:34:10.390061+0000","last_unstale":"2024-09-10T18:36:09.304256+0000","last_undegraded":"2024-09-10T18:36:09.304256+0000","last_fullsized":"2024-09-10T18:36:09.304256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:59.161343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297270+0000","last_change":"2024-09-10T18:34:14.759446+0000","last_active":"2024-09-10T18:36:09.297270+0000","last_peered":"2024-09-10T18:36:09.297270+0000","last_clean":"2024-09-10T18:36:09.297270+0000","last_became_active":"2024-09-10T18:34:14.758968+0000","last_became_peered":"2024-09-10T18:34:14.758968+0000","last_unstale":"2024-09-10T18:36:09.297270+0000","last_undegraded":"2024-09-10T18:36:09.297270+0000","last_fullsized":"2024-09-10T18:36:09.297270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:39.311406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302579+0000","last_change":"2024-09-10T18:34:16.785485+0000","last_active":"2024-09-10T18:36:09.302579+0000","last_peered":"2024-09-10T18:36:09.302579+0000","last_clean":"2024-09-10T18:36:09.302579+0000","last_became_active":"2024-09-10T18:34:16.785128+0000","last_became_peered":"2024-09-10T18:34:16.785128+0000","last_unstale":"2024-09-10T18:36:09.302579+0000","last_undegraded":"2024-09-10T18:36:09.302579+0000","last_fullsized":"2024-09-10T18:36:09.302579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:43.503900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"60'4","reported_seq":26,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.233698+0000","last_change":"2024-09-10T18:34:12.415419+0000","last_active":"2024-09-10T18:35:09.233698+0000","last_peered":"2024-09-10T18:35:09.233698+0000","last_clean":"2024-09-10T18:35:09.233698+0000","last_became_active":"2024-09-10T18:34:12.415127+0000","last_became_peered":"2024-09-10T18:34:12.415127+0000","last_unstale":"2024-09-10T18:35:09.233698+0000","last_undegraded":"2024-09-10T18:35:09.233698+0000","last_fullsized":"2024-09-10T18:35:09.233698+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:17.177881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.234818+0000","last_change":"2024-09-10T18:34:10.377274+0000","last_active":"2024-09-10T18:35:09.234818+0000","last_peered":"2024-09-10T18:35:09.234818+0000","last_clean":"2024-09-10T18:35:09.234818+0000","last_became_active":"2024-09-10T18:34:10.376958+0000","last_became_peered":"2024-09-10T18:34:10.376958+0000","last_unstale":"2024-09-10T18:35:09.234818+0000","last_undegraded":"2024-09-10T18:35:09.234818+0000","last_fullsized":"2024-09-10T18:35:09.234818+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:30.048373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304971+0000","last_change":"2024-09-10T18:34:14.761136+0000","last_active":"2024-09-10T18:36:09.304971+0000","last_peered":"2024-09-10T18:36:09.304971+0000","last_clean":"2024-09-10T18:36:09.304971+0000","last_became_active":"2024-09-10T18:34:14.760817+0000","last_became_peered":"2024-09-10T18:34:14.760817+0000","last_unstale":"2024-09-10T18:36:09.304971+0000","last_undegraded":"2024-09-10T18:36:09.304971+0000","last_fullsized":"2024-09-10T18:36:09.304971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:19.634840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292575+0000","last_change":"2024-09-10T18:34:17.062445+0000","last_active":"2024-09-10T18:36:09.292575+0000","last_peered":"2024-09-10T18:36:09.292575+0000","last_clean":"2024-09-10T18:36:09.292575+0000","last_became_active":"2024-09-10T18:34:17.061610+0000","last_became_peered":"2024-09-10T18:34:17.061610+0000","last_unstale":"2024-09-10T18:36:09.292575+0000","last_undegraded":"2024-09-10T18:36:09.292575+0000","last_fullsized":"2024-09-10T18:36:09.292575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:42.160284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292571+0000","last_change":"2024-09-10T18:34:12.527974+0000","last_active":"2024-09-10T18:36:09.292571+0000","last_peered":"2024-09-10T18:36:09.292571+0000","last_clean":"2024-09-10T18:36:09.292571+0000","last_became_active":"2024-09-10T18:34:12.526963+0000","last_became_peered":"2024-09-10T18:34:12.526963+0000","last_unstale":"2024-09-10T18:36:09.292571+0000","last_undegraded":"2024-09-10T18:36:09.292571+0000","last_fullsized":"2024-09-10T18:36:09.292571+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:20.824387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294627+0000","last_change":"2024-09-10T18:34:10.383808+0000","last_active":"2024-09-10T18:36:09.294627+0000","last_peered":"2024-09-10T18:36:09.294627+0000","last_clean":"2024-09-10T18:36:09.294627+0000","last_became_active":"2024-09-10T18:34:10.383378+0000","last_became_peered":"2024-09-10T18:34:10.383378+0000","last_unstale":"2024-09-10T18:36:09.294627+0000","last_undegraded":"2024-09-10T18:36:09.294627+0000","last_fullsized":"2024-09-10T18:36:09.294627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:54.958532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"60'11","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.846466+0000","last_change":"2024-09-10T18:34:14.760088+0000","last_active":"2024-09-10T18:36:58.846466+0000","last_peered":"2024-09-10T18:36:58.846466+0000","last_clean":"2024-09-10T18:36:58.846466+0000","last_became_active":"2024-09-10T18:34:14.759776+0000","last_became_peered":"2024-09-10T18:34:14.759776+0000","last_unstale":"2024-09-10T18:36:58.846466+0000","last_undegraded":"2024-09-10T18:36:58.846466+0000","last_fullsized":"2024-09-10T18:36:58.846466+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:08.637410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297014+0000","last_change":"2024-09-10T18:34:16.789419+0000","last_active":"2024-09-10T18:36:09.297014+0000","last_peered":"2024-09-10T18:36:09.297014+0000","last_clean":"2024-09-10T18:36:09.297014+0000","last_became_active":"2024-09-10T18:34:16.789098+0000","last_became_peered":"2024-09-10T18:34:16.789098+0000","last_unstale":"2024-09-10T18:36:09.297014+0000","last_undegraded":"2024-09-10T18:36:09.297014+0000","last_fullsized":"2024-09-10T18:36:09.297014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:04.952267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297111+0000","last_change":"2024-09-10T18:34:12.424471+0000","last_active":"2024-09-10T18:36:09.297111+0000","last_peered":"2024-09-10T18:36:09.297111+0000","last_clean":"2024-09-10T18:36:09.297111+0000","last_became_active":"2024-09-10T18:34:12.424068+0000","last_became_peered":"2024-09-10T18:34:12.424068+0000","last_unstale":"2024-09-10T18:36:09.297111+0000","last_undegraded":"2024-09-10T18:36:09.297111+0000","last_fullsized":"2024-09-10T18:36:09.297111+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:52.926536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298525+0000","last_change":"2024-09-10T18:34:10.397588+0000","last_active":"2024-09-10T18:36:09.298525+0000","last_peered":"2024-09-10T18:36:09.298525+0000","last_clean":"2024-09-10T18:36:09.298525+0000","last_became_active":"2024-09-10T18:34:10.397303+0000","last_became_peered":"2024-09-10T18:34:10.397303+0000","last_unstale":"2024-09-10T18:36:09.298525+0000","last_undegraded":"2024-09-10T18:36:09.298525+0000","last_fullsized":"2024-09-10T18:36:09.298525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:45.215055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'11","reported_seq":64,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.832547+0000","last_change":"2024-09-10T18:34:14.753975+0000","last_active":"2024-09-10T18:36:58.832547+0000","last_peered":"2024-09-10T18:36:58.832547+0000","last_clean":"2024-09-10T18:36:58.832547+0000","last_became_active":"2024-09-10T18:34:14.751285+0000","last_became_peered":"2024-09-10T18:34:14.751285+0000","last_unstale":"2024-09-10T18:36:58.832547+0000","last_undegraded":"2024-09-10T18:36:58.832547+0000","last_fullsized":"2024-09-10T18:36:58.832547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:59.245646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285310+0000","last_change":"2024-09-10T18:34:16.772297+0000","last_active":"2024-09-10T18:36:09.285310+0000","last_peered":"2024-09-10T18:36:09.285310+0000","last_clean":"2024-09-10T18:36:09.285310+0000","last_became_active":"2024-09-10T18:34:16.771934+0000","last_became_peered":"2024-09-10T18:34:16.771934+0000","last_unstale":"2024-09-10T18:36:09.285310+0000","last_undegraded":"2024-09-10T18:36:09.285310+0000","last_fullsized":"2024-09-10T18:36:09.285310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:02.348334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292895+0000","last_change":"2024-09-10T18:34:12.410280+0000","last_active":"2024-09-10T18:36:09.292895+0000","last_peered":"2024-09-10T18:36:09.292895+0000","last_clean":"2024-09-10T18:36:09.292895+0000","last_became_active":"2024-09-10T18:34:12.409884+0000","last_became_peered":"2024-09-10T18:34:12.409884+0000","last_unstale":"2024-09-10T18:36:09.292895+0000","last_undegraded":"2024-09-10T18:36:09.292895+0000","last_fullsized":"2024-09-10T18:36:09.292895+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:13.201512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235933+0000","last_change":"2024-09-10T18:34:10.384141+0000","last_active":"2024-09-10T18:35:09.235933+0000","last_peered":"2024-09-10T18:35:09.235933+0000","last_clean":"2024-09-10T18:35:09.235933+0000","last_became_active":"2024-09-10T18:34:10.383308+0000","last_became_peered":"2024-09-10T18:34:10.383308+0000","last_unstale":"2024-09-10T18:35:09.235933+0000","last_undegraded":"2024-09-10T18:35:09.235933+0000","last_fullsized":"2024-09-10T18:35:09.235933+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:53.745126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.299069+0000","last_change":"2024-09-10T18:34:14.762234+0000","last_active":"2024-09-10T18:36:09.299069+0000","last_peered":"2024-09-10T18:36:09.299069+0000","last_clean":"2024-09-10T18:36:09.299069+0000","last_became_active":"2024-09-10T18:34:14.761997+0000","last_became_peered":"2024-09-10T18:34:14.761997+0000","last_unstale":"2024-09-10T18:36:09.299069+0000","last_undegraded":"2024-09-10T18:36:09.299069+0000","last_fullsized":"2024-09-10T18:36:09.299069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:18.581518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.295845+0000","last_change":"2024-09-10T18:34:16.788187+0000","last_active":"2024-09-10T18:36:09.295845+0000","last_peered":"2024-09-10T18:36:09.295845+0000","last_clean":"2024-09-10T18:36:09.295845+0000","last_became_active":"2024-09-10T18:34:16.787854+0000","last_became_peered":"2024-09-10T18:34:16.787854+0000","last_unstale":"2024-09-10T18:36:09.295845+0000","last_undegraded":"2024-09-10T18:36:09.295845+0000","last_fullsized":"2024-09-10T18:36:09.295845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:13.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284952+0000","last_change":"2024-09-10T18:34:12.413814+0000","last_active":"2024-09-10T18:36:09.284952+0000","last_peered":"2024-09-10T18:36:09.284952+0000","last_clean":"2024-09-10T18:36:09.284952+0000","last_became_active":"2024-09-10T18:34:12.413512+0000","last_became_peered":"2024-09-10T18:34:12.413512+0000","last_unstale":"2024-09-10T18:36:09.284952+0000","last_undegraded":"2024-09-10T18:36:09.284952+0000","last_fullsized":"2024-09-10T18:36:09.284952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:58.191220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301440+0000","last_change":"2024-09-10T18:34:10.389420+0000","last_active":"2024-09-10T18:36:09.301440+0000","last_peered":"2024-09-10T18:36:09.301440+0000","last_clean":"2024-09-10T18:36:09.301440+0000","last_became_active":"2024-09-10T18:34:10.389098+0000","last_became_peered":"2024-09-10T18:34:10.389098+0000","last_unstale":"2024-09-10T18:36:09.301440+0000","last_undegraded":"2024-09-10T18:36:09.301440+0000","last_fullsized":"2024-09-10T18:36:09.301440+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:07.696670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285659+0000","last_change":"2024-09-10T18:34:14.758307+0000","last_active":"2024-09-10T18:36:09.285659+0000","last_peered":"2024-09-10T18:36:09.285659+0000","last_clean":"2024-09-10T18:36:09.285659+0000","last_became_active":"2024-09-10T18:34:14.757967+0000","last_became_peered":"2024-09-10T18:34:14.757967+0000","last_unstale":"2024-09-10T18:36:09.285659+0000","last_undegraded":"2024-09-10T18:36:09.285659+0000","last_fullsized":"2024-09-10T18:36:09.285659+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:41.872278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302566+0000","last_change":"2024-09-10T18:34:16.788440+0000","last_active":"2024-09-10T18:36:09.302566+0000","last_peered":"2024-09-10T18:36:09.302566+0000","last_clean":"2024-09-10T18:36:09.302566+0000","last_became_active":"2024-09-10T18:34:16.788171+0000","last_became_peered":"2024-09-10T18:34:16.788171+0000","last_unstale":"2024-09-10T18:36:09.302566+0000","last_undegraded":"2024-09-10T18:36:09.302566+0000","last_fullsized":"2024-09-10T18:36:09.302566+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:40.980577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'6","reported_seq":30,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.299313+0000","last_change":"2024-09-10T18:34:12.429193+0000","last_active":"2024-09-10T18:36:09.299313+0000","last_peered":"2024-09-10T18:36:09.299313+0000","last_clean":"2024-09-10T18:36:09.299313+0000","last_became_active":"2024-09-10T18:34:12.428682+0000","last_became_peered":"2024-09-10T18:34:12.428682+0000","last_unstale":"2024-09-10T18:36:09.299313+0000","last_undegraded":"2024-09-10T18:36:09.299313+0000","last_fullsized":"2024-09-10T18:36:09.299313+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:07.319728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294547+0000","last_change":"2024-09-10T18:34:10.387597+0000","last_active":"2024-09-10T18:36:09.294547+0000","last_peered":"2024-09-10T18:36:09.294547+0000","last_clean":"2024-09-10T18:36:09.294547+0000","last_became_active":"2024-09-10T18:34:10.387347+0000","last_became_peered":"2024-09-10T18:34:10.387347+0000","last_unstale":"2024-09-10T18:36:09.294547+0000","last_undegraded":"2024-09-10T18:36:09.294547+0000","last_fullsized":"2024-09-10T18:36:09.294547+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:13.275399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302254+0000","last_change":"2024-09-10T18:34:14.761053+0000","last_active":"2024-09-10T18:36:09.302254+0000","last_peered":"2024-09-10T18:36:09.302254+0000","last_clean":"2024-09-10T18:36:09.302254+0000","last_became_active":"2024-09-10T18:34:14.760739+0000","last_became_peered":"2024-09-10T18:34:14.760739+0000","last_unstale":"2024-09-10T18:36:09.302254+0000","last_undegraded":"2024-09-10T18:36:09.302254+0000","last_fullsized":"2024-09-10T18:36:09.302254+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:47.732711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292010+0000","last_change":"2024-09-10T18:34:16.778948+0000","last_active":"2024-09-10T18:36:09.292010+0000","last_peered":"2024-09-10T18:36:09.292010+0000","last_clean":"2024-09-10T18:36:09.292010+0000","last_became_active":"2024-09-10T18:34:16.778597+0000","last_became_peered":"2024-09-10T18:34:16.778597+0000","last_unstale":"2024-09-10T18:36:09.292010+0000","last_undegraded":"2024-09-10T18:36:09.292010+0000","last_fullsized":"2024-09-10T18:36:09.292010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:08.778306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294415+0000","last_change":"2024-09-10T18:34:12.413340+0000","last_active":"2024-09-10T18:36:09.294415+0000","last_peered":"2024-09-10T18:36:09.294415+0000","last_clean":"2024-09-10T18:36:09.294415+0000","last_became_active":"2024-09-10T18:34:12.412768+0000","last_became_peered":"2024-09-10T18:34:12.412768+0000","last_unstale":"2024-09-10T18:36:09.294415+0000","last_undegraded":"2024-09-10T18:36:09.294415+0000","last_fullsized":"2024-09-10T18:36:09.294415+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:32.071499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.237334+0000","last_change":"2024-09-10T18:34:10.378473+0000","last_active":"2024-09-10T18:35:09.237334+0000","last_peered":"2024-09-10T18:35:09.237334+0000","last_clean":"2024-09-10T18:35:09.237334+0000","last_became_active":"2024-09-10T18:34:10.378166+0000","last_became_peered":"2024-09-10T18:34:10.378166+0000","last_unstale":"2024-09-10T18:35:09.237334+0000","last_undegraded":"2024-09-10T18:35:09.237334+0000","last_fullsized":"2024-09-10T18:35:09.237334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:29.996352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302715+0000","last_change":"2024-09-10T18:34:14.757400+0000","last_active":"2024-09-10T18:36:09.302715+0000","last_peered":"2024-09-10T18:36:09.302715+0000","last_clean":"2024-09-10T18:36:09.302715+0000","last_became_active":"2024-09-10T18:34:14.756767+0000","last_became_peered":"2024-09-10T18:34:14.756767+0000","last_unstale":"2024-09-10T18:36:09.302715+0000","last_undegraded":"2024-09-10T18:36:09.302715+0000","last_fullsized":"2024-09-10T18:36:09.302715+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:43.490199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304020+0000","last_change":"2024-09-10T18:34:17.073902+0000","last_active":"2024-09-10T18:36:09.304020+0000","last_peered":"2024-09-10T18:36:09.304020+0000","last_clean":"2024-09-10T18:36:09.304020+0000","last_became_active":"2024-09-10T18:34:17.073029+0000","last_became_peered":"2024-09-10T18:34:17.073029+0000","last_unstale":"2024-09-10T18:36:09.304020+0000","last_undegraded":"2024-09-10T18:36:09.304020+0000","last_fullsized":"2024-09-10T18:36:09.304020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:28.392682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292616+0000","last_change":"2024-09-10T18:34:16.780401+0000","last_active":"2024-09-10T18:36:09.292616+0000","last_peered":"2024-09-10T18:36:09.292616+0000","last_clean":"2024-09-10T18:36:09.292616+0000","last_became_active":"2024-09-10T18:34:16.780072+0000","last_became_peered":"2024-09-10T18:34:16.780072+0000","last_unstale":"2024-09-10T18:36:09.292616+0000","last_undegraded":"2024-09-10T18:36:09.292616+0000","last_fullsized":"2024-09-10T18:36:09.292616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.732582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303583+0000","last_change":"2024-09-10T18:34:12.417300+0000","last_active":"2024-09-10T18:36:09.303583+0000","last_peered":"2024-09-10T18:36:09.303583+0000","last_clean":"2024-09-10T18:36:09.303583+0000","last_became_active":"2024-09-10T18:34:12.416444+0000","last_became_peered":"2024-09-10T18:34:12.416444+0000","last_unstale":"2024-09-10T18:36:09.303583+0000","last_undegraded":"2024-09-10T18:36:09.303583+0000","last_fullsized":"2024-09-10T18:36:09.303583+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:36.223910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"53'1","reported_seq":35,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293771+0000","last_change":"2024-09-10T18:34:10.379655+0000","last_active":"2024-09-10T18:36:09.293771+0000","last_peered":"2024-09-10T18:36:09.293771+0000","last_clean":"2024-09-10T18:36:09.293771+0000","last_became_active":"2024-09-10T18:34:10.379349+0000","last_became_peered":"2024-09-10T18:34:10.379349+0000","last_unstale":"2024-09-10T18:36:09.293771+0000","last_undegraded":"2024-09-10T18:36:09.293771+0000","last_fullsized":"2024-09-10T18:36:09.293771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:04.232115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297144+0000","last_change":"2024-09-10T18:34:14.762873+0000","last_active":"2024-09-10T18:36:09.297144+0000","last_peered":"2024-09-10T18:36:09.297144+0000","last_clean":"2024-09-10T18:36:09.297144+0000","last_became_active":"2024-09-10T18:34:14.762609+0000","last_became_peered":"2024-09-10T18:34:14.762609+0000","last_unstale":"2024-09-10T18:36:09.297144+0000","last_undegraded":"2024-09-10T18:36:09.297144+0000","last_fullsized":"2024-09-10T18:36:09.297144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:53.771040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301221+0000","last_change":"2024-09-10T18:34:16.794863+0000","last_active":"2024-09-10T18:36:09.301221+0000","last_peered":"2024-09-10T18:36:09.301221+0000","last_clean":"2024-09-10T18:36:09.301221+0000","last_became_active":"2024-09-10T18:34:16.794619+0000","last_became_peered":"2024-09-10T18:34:16.794619+0000","last_unstale":"2024-09-10T18:36:09.301221+0000","last_undegraded":"2024-09-10T18:36:09.301221+0000","last_fullsized":"2024-09-10T18:36:09.301221+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:04.189834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302387+0000","last_change":"2024-09-10T18:34:12.421836+0000","last_active":"2024-09-10T18:36:09.302387+0000","last_peered":"2024-09-10T18:36:09.302387+0000","last_clean":"2024-09-10T18:36:09.302387+0000","last_became_active":"2024-09-10T18:34:12.421554+0000","last_became_peered":"2024-09-10T18:34:12.421554+0000","last_unstale":"2024-09-10T18:36:09.302387+0000","last_undegraded":"2024-09-10T18:36:09.302387+0000","last_fullsized":"2024-09-10T18:36:09.302387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:41.279161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303668+0000","last_change":"2024-09-10T18:34:10.396213+0000","last_active":"2024-09-10T18:36:09.303668+0000","last_peered":"2024-09-10T18:36:09.303668+0000","last_clean":"2024-09-10T18:36:09.303668+0000","last_became_active":"2024-09-10T18:34:10.395688+0000","last_became_peered":"2024-09-10T18:34:10.395688+0000","last_unstale":"2024-09-10T18:36:09.303668+0000","last_undegraded":"2024-09-10T18:36:09.303668+0000","last_fullsized":"2024-09-10T18:36:09.303668+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:35.077708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":69,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:58.832499+0000","last_change":"2024-09-10T18:34:14.758358+0000","last_active":"2024-09-10T18:36:58.832499+0000","last_peered":"2024-09-10T18:36:58.832499+0000","last_clean":"2024-09-10T18:36:58.832499+0000","last_became_active":"2024-09-10T18:34:14.756787+0000","last_became_peered":"2024-09-10T18:34:14.756787+0000","last_unstale":"2024-09-10T18:36:58.832499+0000","last_undegraded":"2024-09-10T18:36:58.832499+0000","last_fullsized":"2024-09-10T18:36:58.832499+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:20.612765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285015+0000","last_change":"2024-09-10T18:34:17.059458+0000","last_active":"2024-09-10T18:36:09.285015+0000","last_peered":"2024-09-10T18:36:09.285015+0000","last_clean":"2024-09-10T18:36:09.285015+0000","last_became_active":"2024-09-10T18:34:17.058421+0000","last_became_peered":"2024-09-10T18:34:17.058421+0000","last_unstale":"2024-09-10T18:36:09.285015+0000","last_undegraded":"2024-09-10T18:36:09.285015+0000","last_fullsized":"2024-09-10T18:36:09.285015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:07.162012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.236180+0000","last_change":"2024-09-10T18:34:10.379347+0000","last_active":"2024-09-10T18:35:09.236180+0000","last_peered":"2024-09-10T18:35:09.236180+0000","last_clean":"2024-09-10T18:35:09.236180+0000","last_became_active":"2024-09-10T18:34:10.379067+0000","last_became_peered":"2024-09-10T18:34:10.379067+0000","last_unstale":"2024-09-10T18:35:09.236180+0000","last_undegraded":"2024-09-10T18:35:09.236180+0000","last_fullsized":"2024-09-10T18:35:09.236180+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:21.812547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"60'5","reported_seq":31,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298386+0000","last_change":"2024-09-10T18:34:12.424354+0000","last_active":"2024-09-10T18:36:09.298386+0000","last_peered":"2024-09-10T18:36:09.298386+0000","last_clean":"2024-09-10T18:36:09.298386+0000","last_became_active":"2024-09-10T18:34:12.424029+0000","last_became_peered":"2024-09-10T18:34:12.424029+0000","last_unstale":"2024-09-10T18:36:09.298386+0000","last_undegraded":"2024-09-10T18:36:09.298386+0000","last_fullsized":"2024-09-10T18:36:09.298386+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:23.902006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.663583+0000","last_change":"2024-09-10T18:36:10.663583+0000","last_active":"2024-09-10T18:36:10.663583+0000","last_peered":"2024-09-10T18:36:10.663583+0000","last_clean":"2024-09-10T18:36:10.663583+0000","last_became_active":"2024-09-10T18:36:10.662778+0000","last_became_peered":"2024-09-10T18:36:10.662778+0000","last_unstale":"2024-09-10T18:36:10.663583+0000","last_undegraded":"2024-09-10T18:36:10.663583+0000","last_fullsized":"2024-09-10T18:36:10.663583+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:22.082908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304875+0000","last_change":"2024-09-10T18:34:17.073346+0000","last_active":"2024-09-10T18:36:09.304875+0000","last_peered":"2024-09-10T18:36:09.304875+0000","last_clean":"2024-09-10T18:36:09.304875+0000","last_became_active":"2024-09-10T18:34:17.072653+0000","last_became_peered":"2024-09-10T18:34:17.072653+0000","last_unstale":"2024-09-10T18:36:09.304875+0000","last_undegraded":"2024-09-10T18:36:09.304875+0000","last_fullsized":"2024-09-10T18:36:09.304875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:33.753289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305255+0000","last_change":"2024-09-10T18:34:10.388032+0000","last_active":"2024-09-10T18:36:09.305255+0000","last_peered":"2024-09-10T18:36:09.305255+0000","last_clean":"2024-09-10T18:36:09.305255+0000","last_became_active":"2024-09-10T18:34:10.387707+0000","last_became_peered":"2024-09-10T18:34:10.387707+0000","last_unstale":"2024-09-10T18:36:09.305255+0000","last_undegraded":"2024-09-10T18:36:09.305255+0000","last_fullsized":"2024-09-10T18:36:09.305255+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:40.885222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284506+0000","last_change":"2024-09-10T18:34:12.531543+0000","last_active":"2024-09-10T18:36:09.284506+0000","last_peered":"2024-09-10T18:36:09.284506+0000","last_clean":"2024-09-10T18:36:09.284506+0000","last_became_active":"2024-09-10T18:34:12.530541+0000","last_became_peered":"2024-09-10T18:34:12.530541+0000","last_unstale":"2024-09-10T18:36:09.284506+0000","last_undegraded":"2024-09-10T18:36:09.284506+0000","last_fullsized":"2024-09-10T18:36:09.284506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:37.855220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303593+0000","last_change":"2024-09-10T18:34:14.769494+0000","last_active":"2024-09-10T18:36:09.303593+0000","last_peered":"2024-09-10T18:36:09.303593+0000","last_clean":"2024-09-10T18:36:09.303593+0000","last_became_active":"2024-09-10T18:34:14.769231+0000","last_became_peered":"2024-09-10T18:34:14.769231+0000","last_unstale":"2024-09-10T18:36:09.303593+0000","last_undegraded":"2024-09-10T18:36:09.303593+0000","last_fullsized":"2024-09-10T18:36:09.303593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:40.353540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":149,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298620,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446532,"statfs":{"total":95995035648,"available":95689248768,"internally_reserved":0,"allocated":1294336,"data_stored":855742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.56799999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.29,"5min":1.431,"15min":1.431},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.89100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0489999999999999,"5min":1.214,"15min":1.214},"last":1.137}]},{"osd":1,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.96499999999999997,"15min":0.94299999999999995},"min":{"1min":0.77100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4039999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.214,"5min":1.677,"15min":1.677},"last":0.98199999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.622,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3120000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.107},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.94799999999999995,"15min":0.95999999999999996},"min":{"1min":0.69799999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1899999999999999,"5min":1.323,"15min":1.323},"last":0.95099999999999996}]},{"osd":3,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":0.94699999999999995},"min":{"1min":0.69299999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2669999999999999,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94099999999999995,"15min":0.94199999999999995},"min":{"1min":0.65500000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2350000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.878,"15min":0.876},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.161,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94599999999999995,"15min":0.94999999999999996},"min":{"1min":0.61699999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.212,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.0629999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0680000000000001,"15min":1.0700000000000001},"min":{"1min":0.63300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4299999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":1},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.159,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.0369999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1279999999999999,"15min":1.1299999999999999},"min":{"1min":0.80300000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4890000000000001,"5min":1.595,"15min":1.595},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0640000000000001,"15min":1.0740000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4339999999999999,"5min":1.643,"15min":1.643},"last":1.1639999999999999}]}]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298584,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446568,"statfs":{"total":95995035648,"available":95689285632,"internally_reserved":0,"allocated":1257472,"data_stored":853608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89000000000000001,"15min":0.85799999999999998},"min":{"1min":0.57699999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.294,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.875,"15min":0.877},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.0549999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.89400000000000002,"15min":0.91500000000000004},"min":{"1min":0.64800000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.24,"5min":1.488,"15min":1.488},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.94899999999999995,"15min":0.93799999999999994},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.365,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.103}]},{"osd":2,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95699999999999996,"15min":0.94299999999999995},"min":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3340000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.008,"15min":1},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.1359999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":0.88900000000000001},"min":{"1min":0.61299999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.153,"5min":1.327,"15min":1.327},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1,"15min":0.98499999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2490000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.92800000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86699999999999999,"15min":0.85399999999999998},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.216,"5min":1.266,"15min":1.266},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.92900000000000005},"min":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.96999999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.997,"15min":0.97499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3029999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.004,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.82199999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:36:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.135,"15min":1.1739999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.508,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.022},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.141,"15min":1.1579999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.425,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.004}]}]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298012,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447140,"statfs":{"total":95995035648,"available":95689871360,"internally_reserved":0,"allocated":671744,"data_stored":263962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86099999999999999,"15min":0.83099999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2549999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.872,"15min":0.88100000000000001},"min":{"1min":0.64000000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.379,"15min":1.379},"last":0.97099999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89900000000000002,"15min":0.88200000000000001},"min":{"1min":0.64400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.202,"5min":1.226,"15min":1.226},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.63300000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2150000000000001,"5min":1.278,"15min":1.278},"last":0.90700000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.89400000000000002,"15min":0.88300000000000001},"min":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1779999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.054},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.95399999999999996,"15min":0.93300000000000005},"min":{"1min":0.61899999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.323,"5min":1.359,"15min":1.359},"last":0.84499999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.93700000000000006,"15min":0.88900000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.25,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.95499999999999996,"15min":0.94699999999999995},"min":{"1min":0.69899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.27,"5min":1.542,"15min":1.542},"last":0.875}]},{"osd":4,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88900000000000001,"15min":0.83099999999999996},"min":{"1min":0.58699999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3049999999999999,"5min":1.423,"15min":1.423},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.97099999999999997,"15min":0.92300000000000004},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0649999999999999,"15min":1.044},"min":{"1min":0.55900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5660000000000001,"5min":1.629,"15min":1.629},"last":1.117},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0069999999999999,"15min":0.94799999999999995},"min":{"1min":0.71499999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.75900000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:36:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1279999999999999,"15min":1.161},"min":{"1min":0.70299999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.105,"5min":1.1990000000000001,"15min":1.2330000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.508,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.0229999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298056,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447096,"statfs":{"total":95995035648,"available":95689826304,"internally_reserved":0,"allocated":712704,"data_stored":265771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82299999999999995,"15min":0.81499999999999995},"min":{"1min":0.58899999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.137,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.024},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81399999999999995,"15min":0.83599999999999997},"min":{"1min":0.53300000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0609999999999999,"5min":1.163,"15min":1.163},"last":0.82699999999999996}]},{"osd":1,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.86799999999999999,"15min":0.81899999999999995},"min":{"1min":0.75900000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1930000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.874},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.86199999999999999,"15min":0.83399999999999996},"min":{"1min":0.66800000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2310000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.085}]},{"osd":2,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.93200000000000005,"15min":0.89600000000000002},"min":{"1min":0.63600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1439999999999999,"5min":1.216,"15min":1.216},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91200000000000003,"15min":0.89400000000000002},"min":{"1min":0.71099999999999997,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":1.29,"5min":1.405,"15min":1.405},"last":0.92200000000000004}]},{"osd":3,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.92500000000000004,"15min":0.89300000000000002},"min":{"1min":0.65100000000000002,"5min":0.62,"15min":0.62},"max":{"1min":1.1919999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.95599999999999996,"15min":0.93799999999999994},"min":{"1min":0.71299999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.323}]},{"osd":5,"last update":"Tue Sep 10 18:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82699999999999996,"15min":0.752},"min":{"1min":0.64200000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0900000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.048,"5min":0.97699999999999998,"15min":0.91200000000000003},"min":{"1min":0.72299999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3260000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.117}]},{"osd":6,"last update":"Tue Sep 10 18:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.88700000000000001,"15min":0.84499999999999997},"min":{"1min":0.61599999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.86299999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2410000000000001,"5min":1.349,"15min":1.349},"last":1.1519999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.018,"15min":1.0389999999999999},"min":{"1min":0.59799999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.327,"5min":1.329,"15min":1.329},"last":1.202},{"interface":"front","average":{"1min":1.075,"5min":1.085,"15min":1.103},"min":{"1min":0.79600000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.173}]}]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298012,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447140,"statfs":{"total":95995035648,"available":95689871360,"internally_reserved":0,"allocated":671744,"data_stored":263935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39900000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93500000000000005,"5min":0.97299999999999998,"15min":1.819},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.748,"15min":0.72499999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.98899999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.67800000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76100000000000001,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0229999999999999,"5min":1.149,"15min":1.736},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1.0389999999999999,"15min":1.2130000000000001},"last":0.755}]},{"osd":2,"last update":"Tue Sep 10 18:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.77800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.494},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.75800000000000001},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.347},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.95499999999999996,"15min":0.92100000000000004},"min":{"1min":0.64100000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2370000000000001,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":0.91000000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.95199999999999996}]},{"osd":5,"last update":"Tue Sep 10 18:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.94399999999999995},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.4339999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.0129999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.3340000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:36:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0620000000000001,"15min":1.046},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.321,"5min":1.464,"15min":1.464},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.044,"15min":1.0229999999999999},"min":{"1min":0.84299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.365,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:36:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1339999999999999,"15min":1.153},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.452},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.115,"15min":1.135},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.036}]}]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298012,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447140,"statfs":{"total":95995035648,"available":95689871360,"internally_reserved":0,"allocated":675840,"data_stored":264040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.69399999999999995,"15min":0.61299999999999999},"min":{"1min":0.47199999999999998,"5min":0.435,"15min":0.38900000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.62},{"interface":"front","average":{"1min":0.754,"5min":0.72499999999999998,"15min":0.63500000000000001},"min":{"1min":0.50600000000000001,"5min":0.503,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":1.0169999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.67400000000000004},"min":{"1min":0.58099999999999996,"5min":0.48599999999999999,"15min":0.41899999999999998},"max":{"1min":1.224,"5min":1.498,"15min":1.498},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.77100000000000002,"15min":0.69599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0489999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Sep 10 18:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82699999999999996,"15min":0.77200000000000002},"min":{"1min":0.58599999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82799999999999996,"15min":0.77000000000000002},"min":{"1min":0.55000000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2729999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.871}]},{"osd":4,"last update":"Tue Sep 10 18:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.92200000000000004,"15min":0.91000000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.0129999999999999,"15min":0.98199999999999998},"min":{"1min":0.72699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.3879999999999999,"5min":1.5029999999999999,"15min":1.5069999999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96899999999999997,"15min":0.94699999999999995},"min":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.099,"15min":1.0509999999999999},"min":{"1min":0.88400000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.0660000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.135,"15min":1.119},"min":{"1min":0.81399999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.085,"15min":1.0649999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.99299999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.095,"15min":1.0880000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.204,"15min":1.1910000000000001},"min":{"1min":0.94599999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.2090000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298020,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447132,"statfs":{"total":95995035648,"available":95689863168,"internally_reserved":0,"allocated":679936,"data_stored":263737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.65400000000000003,"15min":0.55700000000000005},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.33700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.78000000000000003,"15min":0.61699999999999999},"min":{"1min":0.58399999999999996,"5min":0.45800000000000002,"15min":0.30399999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.5329999999999999},"last":0.84999999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:37:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.72099999999999997,"15min":0.65100000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79600000000000004,"15min":0.69999999999999996},"min":{"1min":0.59399999999999997,"5min":0.501,"15min":0.44},"max":{"1min":1.365,"5min":1.365,"15min":1.367},"last":0.72499999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84399999999999997,"15min":0.78600000000000003},"min":{"1min":0.625,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.4870000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.90000000000000002,"5min":1.028,"15min":1.028},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.91800000000000004,"15min":0.86199999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.61799999999999999},"max":{"1min":1.3300000000000001,"5min":1.583,"15min":1.583},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.093,"5min":1.034,"15min":0.93799999999999994},"min":{"1min":0.73599999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.454,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0580000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.94999999999999996,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.236,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.0840000000000001,"15min":1.05},"min":{"1min":0.82199999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.508,"5min":1.722,"15min":1.722},"last":1.1479999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:36:35 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.133,"15min":1.133},"min":{"1min":0.92500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.302,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3919999999999999,"5min":1.613,"15min":1.613},"last":0.99199999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.96299999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.383,"5min":1.484,"15min":1.484},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0609999999999999,"15min":1.0549999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.3180000000000001,"5min":1.875,"15min":1.875},"last":1.093}]}]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298604,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446548,"statfs":{"total":95995035648,"available":95689265152,"internally_reserved":0,"allocated":1277952,"data_stored":854188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67200000000000004,"15min":0.58899999999999997},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92600000000000005,"5min":1.081,"15min":1.081},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.61799999999999999},"min":{"1min":0.44800000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:36:57 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72699999999999998,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.117,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.628},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81000000000000005,"15min":0.71399999999999997},"min":{"1min":0.57099999999999995,"5min":0.47499999999999998,"15min":0.41599999999999998},"max":{"1min":1.052,"5min":1.1619999999999999,"15min":1.1819999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:36:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79800000000000004,"15min":0.75900000000000001},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88800000000000001,"15min":0.80100000000000005},"min":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.45700000000000002},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.83199999999999996}]},{"osd":4,"last update":"Tue Sep 10 18:36:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.90000000000000002,"15min":0.86099999999999999},"min":{"1min":0.65700000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.0069999999999999,"15min":1.0089999999999999},"min":{"1min":0.70899999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.349,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0249999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.018,"15min":0.96399999999999997},"min":{"1min":0.76500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.252,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.101,"5min":1.0820000000000001,"15min":1.0620000000000001},"min":{"1min":0.82999999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3939999999999999,"5min":1.474,"15min":1.474},"last":1.196}]},{"osd":6,"last update":"Tue Sep 10 18:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.69699999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4219999999999999,"5min":1.548,"15min":1.548},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.141,"15min":1.135},"min":{"1min":0.79000000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.262}]},{"osd":7,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.034,"15min":1.034},"min":{"1min":0.75,"5min":0.5,"15min":0.5},"max":{"1min":1.3700000000000001,"5min":1.458,"15min":1.458},"last":1.232},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1359999999999999,"15min":1.1259999999999999},"min":{"1min":0.81699999999999995,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.425,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.2130000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:37:11.073 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:37:11.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:10 smithi100 bash[18597]: cluster 2024-09-10T18:37:09.279666+0000 mgr.y (mgr.24512) 106 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:11.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:10 smithi100 bash[31607]: cluster 2024-09-10T18:37:09.279666+0000 mgr.y (mgr.24512) 106 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:11.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:10 smithi151 bash[22565]: cluster 2024-09-10T18:37:09.279666+0000 mgr.y (mgr.24512) 106 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:37:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:11] "GET /metrics HTTP/1.1" 200 219675 "" "Prometheus/2.43.0" 2024-09-10T18:37:12.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:11 smithi100 bash[31607]: audit 2024-09-10T18:37:10.309898+0000 mgr.y (mgr.24512) 107 : audit [DBG] from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:37:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:11 smithi100 bash[18597]: audit 2024-09-10T18:37:10.309898+0000 mgr.y (mgr.24512) 107 : audit [DBG] from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:37:12.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:37:11 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:37:12.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:11 smithi151 bash[22565]: audit 2024-09-10T18:37:10.309898+0000 mgr.y (mgr.24512) 107 : audit [DBG] from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:37:13.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:12 smithi100 bash[18597]: cluster 2024-09-10T18:37:11.280502+0000 mgr.y (mgr.24512) 108 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:13.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:12 smithi100 bash[31607]: cluster 2024-09-10T18:37:11.280502+0000 mgr.y (mgr.24512) 108 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:13.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:12 smithi151 bash[22565]: cluster 2024-09-10T18:37:11.280502+0000 mgr.y (mgr.24512) 108 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:13 smithi100 bash[18597]: audit 2024-09-10T18:37:11.976889+0000 mgr.y (mgr.24512) 109 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:13 smithi100 bash[18597]: cluster 2024-09-10T18:37:13.281313+0000 mgr.y (mgr.24512) 110 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:13 smithi100 bash[31607]: audit 2024-09-10T18:37:11.976889+0000 mgr.y (mgr.24512) 109 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:13 smithi100 bash[31607]: cluster 2024-09-10T18:37:13.281313+0000 mgr.y (mgr.24512) 110 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:14.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:13 smithi151 bash[22565]: audit 2024-09-10T18:37:11.976889+0000 mgr.y (mgr.24512) 109 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:14.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:13 smithi151 bash[22565]: cluster 2024-09-10T18:37:13.281313+0000 mgr.y (mgr.24512) 110 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:14.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:37:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:37:15.851 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:37:16.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:16 smithi151 bash[22565]: cluster 2024-09-10T18:37:15.282672+0000 mgr.y (mgr.24512) 111 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:16.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:16 smithi100 bash[18597]: cluster 2024-09-10T18:37:15.282672+0000 mgr.y (mgr.24512) 111 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:16.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:16 smithi100 bash[31607]: cluster 2024-09-10T18:37:15.282672+0000 mgr.y (mgr.24512) 111 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:18.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:18 smithi100 bash[18597]: cluster 2024-09-10T18:37:17.283357+0000 mgr.y (mgr.24512) 112 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:18.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:18 smithi100 bash[31607]: cluster 2024-09-10T18:37:17.283357+0000 mgr.y (mgr.24512) 112 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:18.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:18 smithi151 bash[22565]: cluster 2024-09-10T18:37:17.283357+0000 mgr.y (mgr.24512) 112 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:19.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- 172.21.15.100:0/298461660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3104109050 msgr2=0x7f310410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:19.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 --2- 172.21.15.100:0/298461660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3104109050 0x7f310410b430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009cd0 tx=0x7f30fc004d10 comp rx=0 tx=0).stop 2024-09-10T18:37:19.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- 172.21.15.100:0/298461660 shutdown_connections 2024-09-10T18:37:19.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 --2- 172.21.15.100:0/298461660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3104109050 0x7f310410b430 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 --2- 172.21.15.100:0/298461660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104065800 0x7f3104065c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 --2- 172.21.15.100:0/298461660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31040fd680 0x7f31040652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- 172.21.15.100:0/298461660 >> 172.21.15.100:0/298461660 conn(0x7f3104072440 msgr2=0x7f3104072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:19.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- 172.21.15.100:0/298461660 shutdown_connections 2024-09-10T18:37:19.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- 172.21.15.100:0/298461660 wait complete. 2024-09-10T18:37:19.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 Processor -- start 2024-09-10T18:37:19.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- start start 2024-09-10T18:37:19.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3104065800 0x7f31040fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:19.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31040fd680 0x7f31040fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104109050 0x7f31040fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3104101e30 con 0x7f31040fd680 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310411f6a0 con 0x7f3104065800 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310411f860 con 0x7f3104109050 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104109050 0x7f31040fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3108a44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31040fd680 0x7f31040fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104109050 0x7f31040fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:42130/0 (socket says 172.21.15.100:42130) 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 -- 172.21.15.100:0/4026632140 learned_addr learned my addr 172.21.15.100:0/4026632140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109245640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3104065800 0x7f31040fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 -- 172.21.15.100:0/4026632140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3104065800 msgr2=0x7f31040fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3104065800 0x7f31040fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 -- 172.21.15.100:0/4026632140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31040fd680 msgr2=0x7f31040fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:19.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31040fd680 0x7f31040fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 -- 172.21.15.100:0/4026632140 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0098b0 con 0x7f3104109050 2024-09-10T18:37:19.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f3109a46640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104109050 0x7f31040fe680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30fc02a8d0 tx=0x7f30fc004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:19.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f30f9ffb640 1 -- 172.21.15.100:0/4026632140 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc038070 con 0x7f3104109050 2024-09-10T18:37:19.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f30f9ffb640 1 -- 172.21.15.100:0/4026632140 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f30fc033e30 con 0x7f3104109050 2024-09-10T18:37:19.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310411fae0 con 0x7f3104109050 2024-09-10T18:37:19.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3104120080 con 0x7f3104109050 2024-09-10T18:37:19.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.387+0000 7f30f9ffb640 1 -- 172.21.15.100:0/4026632140 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30fc03ca70 con 0x7f3104109050 2024-09-10T18:37:19.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.391+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31040fbd70 con 0x7f3104109050 2024-09-10T18:37:19.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.391+0000 7f30f9ffb640 1 -- 172.21.15.100:0/4026632140 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f30fc030070 con 0x7f3104109050 2024-09-10T18:37:19.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.391+0000 7f30f9ffb640 1 --2- 172.21.15.100:0/4026632140 >> v2:172.21.15.100:6800/2155730476 conn(0x7f30f006eec0 0x7f30f0071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:19.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.391+0000 7f30f9ffb640 1 -- 172.21.15.100:0/4026632140 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f30fc0aed30 con 0x7f3104109050 2024-09-10T18:37:19.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.391+0000 7f3109245640 1 --2- 172.21.15.100:0/4026632140 >> v2:172.21.15.100:6800/2155730476 conn(0x7f30f006eec0 0x7f30f0071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:19.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.391+0000 7f3109245640 1 --2- 172.21.15.100:0/4026632140 >> v2:172.21.15.100:6800/2155730476 conn(0x7f30f006eec0 0x7f30f0071370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f30f4004eb0 tx=0x7f30f400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:19.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.399+0000 7f30f9ffb640 1 -- 172.21.15.100:0/4026632140 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f30fc07bc50 con 0x7f3104109050 2024-09-10T18:37:19.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.571+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 --> v2:172.21.15.100:6800/2155730476 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f310410dee0 con 0x7f30f006eec0 2024-09-10T18:37:19.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.583+0000 7f30f9ffb640 1 -- 172.21.15.100:0/4026632140 <== mgr.24512 v2:172.21.15.100:6800/2155730476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+382187 (secure 0 0 0) 0x7f310410dee0 con 0x7f30f006eec0 2024-09-10T18:37:19.586 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:37:19.590 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-10T18:37:19.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 >> v2:172.21.15.100:6800/2155730476 conn(0x7f30f006eec0 msgr2=0x7f30f0071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:19.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 --2- 172.21.15.100:0/4026632140 >> v2:172.21.15.100:6800/2155730476 conn(0x7f30f006eec0 0x7f30f0071370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f30f4004eb0 tx=0x7f30f400b040 comp rx=0 tx=0).stop 2024-09-10T18:37:19.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104109050 msgr2=0x7f31040fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:19.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104109050 0x7f31040fe680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30fc02a8d0 tx=0x7f30fc004d10 comp rx=0 tx=0).stop 2024-09-10T18:37:19.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 shutdown_connections 2024-09-10T18:37:19.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 --2- 172.21.15.100:0/4026632140 >> v2:172.21.15.100:6800/2155730476 conn(0x7f30f006eec0 0x7f30f0071370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3104109050 0x7f31040fe680 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31040fd680 0x7f31040fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 --2- 172.21.15.100:0/4026632140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3104065800 0x7f31040fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:19.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 >> 172.21.15.100:0/4026632140 conn(0x7f3104072440 msgr2=0x7f31041097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:19.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 shutdown_connections 2024-09-10T18:37:19.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:19.591+0000 7f310b4a9640 1 -- 172.21.15.100:0/4026632140 wait complete. 2024-09-10T18:37:20.277 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-10T18:37:19.283658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1033,"num_read_kb":889,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":574,"ondisk_log_size":574,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385920,"kb_used_data":7072,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575296,"statfs":{"total":767960285184,"available":765517103104,"internally_reserved":0,"allocated":7241728,"data_stored":3884983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005038"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303531+0000","last_change":"2024-09-10T18:34:17.075989+0000","last_active":"2024-09-10T18:36:09.303531+0000","last_peered":"2024-09-10T18:36:09.303531+0000","last_clean":"2024-09-10T18:36:09.303531+0000","last_became_active":"2024-09-10T18:34:17.075423+0000","last_became_peered":"2024-09-10T18:34:17.075423+0000","last_unstale":"2024-09-10T18:36:09.303531+0000","last_undegraded":"2024-09-10T18:36:09.303531+0000","last_fullsized":"2024-09-10T18:36:09.303531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:07.113580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301433+0000","last_change":"2024-09-10T18:34:10.395850+0000","last_active":"2024-09-10T18:36:09.301433+0000","last_peered":"2024-09-10T18:36:09.301433+0000","last_clean":"2024-09-10T18:36:09.301433+0000","last_became_active":"2024-09-10T18:34:10.395374+0000","last_became_peered":"2024-09-10T18:34:10.395374+0000","last_unstale":"2024-09-10T18:36:09.301433+0000","last_undegraded":"2024-09-10T18:36:09.301433+0000","last_fullsized":"2024-09-10T18:36:09.301433+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:39.666540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304665+0000","last_change":"2024-09-10T18:34:12.541967+0000","last_active":"2024-09-10T18:36:09.304665+0000","last_peered":"2024-09-10T18:36:09.304665+0000","last_clean":"2024-09-10T18:36:09.304665+0000","last_became_active":"2024-09-10T18:34:12.541697+0000","last_became_peered":"2024-09-10T18:34:12.541697+0000","last_unstale":"2024-09-10T18:36:09.304665+0000","last_undegraded":"2024-09-10T18:36:09.304665+0000","last_fullsized":"2024-09-10T18:36:09.304665+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:07.563690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303071+0000","last_change":"2024-09-10T18:34:14.757787+0000","last_active":"2024-09-10T18:36:09.303071+0000","last_peered":"2024-09-10T18:36:09.303071+0000","last_clean":"2024-09-10T18:36:09.303071+0000","last_became_active":"2024-09-10T18:34:14.757444+0000","last_became_peered":"2024-09-10T18:34:14.757444+0000","last_unstale":"2024-09-10T18:36:09.303071+0000","last_undegraded":"2024-09-10T18:36:09.303071+0000","last_fullsized":"2024-09-10T18:36:09.303071+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:55.055599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304583+0000","last_change":"2024-09-10T18:34:10.394278+0000","last_active":"2024-09-10T18:36:09.304583+0000","last_peered":"2024-09-10T18:36:09.304583+0000","last_clean":"2024-09-10T18:36:09.304583+0000","last_became_active":"2024-09-10T18:34:10.393723+0000","last_became_peered":"2024-09-10T18:34:10.393723+0000","last_unstale":"2024-09-10T18:36:09.304583+0000","last_undegraded":"2024-09-10T18:36:09.304583+0000","last_fullsized":"2024-09-10T18:36:09.304583+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:02.419785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298020+0000","last_change":"2024-09-10T18:34:12.543977+0000","last_active":"2024-09-10T18:36:09.298020+0000","last_peered":"2024-09-10T18:36:09.298020+0000","last_clean":"2024-09-10T18:36:09.298020+0000","last_became_active":"2024-09-10T18:34:12.543137+0000","last_became_peered":"2024-09-10T18:34:12.543137+0000","last_unstale":"2024-09-10T18:36:09.298020+0000","last_undegraded":"2024-09-10T18:36:09.298020+0000","last_fullsized":"2024-09-10T18:36:09.298020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:01.811278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283643+0000","last_change":"2024-09-10T18:34:14.754705+0000","last_active":"2024-09-10T18:36:09.283643+0000","last_peered":"2024-09-10T18:36:09.283643+0000","last_clean":"2024-09-10T18:36:09.283643+0000","last_became_active":"2024-09-10T18:34:14.754351+0000","last_became_peered":"2024-09-10T18:34:14.754351+0000","last_unstale":"2024-09-10T18:36:09.283643+0000","last_undegraded":"2024-09-10T18:36:09.283643+0000","last_fullsized":"2024-09-10T18:36:09.283643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:55.227711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283929+0000","last_change":"2024-09-10T18:34:16.774627+0000","last_active":"2024-09-10T18:36:09.283929+0000","last_peered":"2024-09-10T18:36:09.283929+0000","last_clean":"2024-09-10T18:36:09.283929+0000","last_became_active":"2024-09-10T18:34:16.774183+0000","last_became_peered":"2024-09-10T18:34:16.774183+0000","last_unstale":"2024-09-10T18:36:09.283929+0000","last_undegraded":"2024-09-10T18:36:09.283929+0000","last_fullsized":"2024-09-10T18:36:09.283929+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:54.090471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293199+0000","last_change":"2024-09-10T18:34:10.386014+0000","last_active":"2024-09-10T18:36:09.293199+0000","last_peered":"2024-09-10T18:36:09.293199+0000","last_clean":"2024-09-10T18:36:09.293199+0000","last_became_active":"2024-09-10T18:34:10.385139+0000","last_became_peered":"2024-09-10T18:34:10.385139+0000","last_unstale":"2024-09-10T18:36:09.293199+0000","last_undegraded":"2024-09-10T18:36:09.293199+0000","last_fullsized":"2024-09-10T18:36:09.293199+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:10.781879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294065+0000","last_change":"2024-09-10T18:34:12.412668+0000","last_active":"2024-09-10T18:36:09.294065+0000","last_peered":"2024-09-10T18:36:09.294065+0000","last_clean":"2024-09-10T18:36:09.294065+0000","last_became_active":"2024-09-10T18:34:12.412250+0000","last_became_peered":"2024-09-10T18:34:12.412250+0000","last_unstale":"2024-09-10T18:36:09.294065+0000","last_undegraded":"2024-09-10T18:36:09.294065+0000","last_fullsized":"2024-09-10T18:36:09.294065+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:29.440098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.296554+0000","last_change":"2024-09-10T18:36:10.296554+0000","last_active":"2024-09-10T18:36:10.296554+0000","last_peered":"2024-09-10T18:36:10.296554+0000","last_clean":"2024-09-10T18:36:10.296554+0000","last_became_active":"2024-09-10T18:36:10.295722+0000","last_became_peered":"2024-09-10T18:36:10.295722+0000","last_unstale":"2024-09-10T18:36:10.296554+0000","last_undegraded":"2024-09-10T18:36:10.296554+0000","last_fullsized":"2024-09-10T18:36:10.296554+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:43.381641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"60'1","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293881+0000","last_change":"2024-09-10T18:34:16.777489+0000","last_active":"2024-09-10T18:36:09.293881+0000","last_peered":"2024-09-10T18:36:09.293881+0000","last_clean":"2024-09-10T18:36:09.293881+0000","last_became_active":"2024-09-10T18:34:16.777161+0000","last_became_peered":"2024-09-10T18:34:16.777161+0000","last_unstale":"2024-09-10T18:36:09.293881+0000","last_undegraded":"2024-09-10T18:36:09.293881+0000","last_fullsized":"2024-09-10T18:36:09.293881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:45.516483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292998+0000","last_change":"2024-09-10T18:34:10.383837+0000","last_active":"2024-09-10T18:36:09.292998+0000","last_peered":"2024-09-10T18:36:09.292998+0000","last_clean":"2024-09-10T18:36:09.292998+0000","last_became_active":"2024-09-10T18:34:10.383543+0000","last_became_peered":"2024-09-10T18:34:10.383543+0000","last_unstale":"2024-09-10T18:36:09.292998+0000","last_undegraded":"2024-09-10T18:36:09.292998+0000","last_fullsized":"2024-09-10T18:36:09.292998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.480466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"60'12","reported_seq":44,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294816+0000","last_change":"2024-09-10T18:34:12.415075+0000","last_active":"2024-09-10T18:36:09.294816+0000","last_peered":"2024-09-10T18:36:09.294816+0000","last_clean":"2024-09-10T18:36:09.294816+0000","last_became_active":"2024-09-10T18:34:12.414803+0000","last_became_peered":"2024-09-10T18:34:12.414803+0000","last_unstale":"2024-09-10T18:36:09.294816+0000","last_undegraded":"2024-09-10T18:36:09.294816+0000","last_fullsized":"2024-09-10T18:36:09.294816+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:05.195754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284259+0000","last_change":"2024-09-10T18:34:14.755453+0000","last_active":"2024-09-10T18:36:09.284259+0000","last_peered":"2024-09-10T18:36:09.284259+0000","last_clean":"2024-09-10T18:36:09.284259+0000","last_became_active":"2024-09-10T18:34:14.755158+0000","last_became_peered":"2024-09-10T18:34:14.755158+0000","last_unstale":"2024-09-10T18:36:09.284259+0000","last_undegraded":"2024-09-10T18:36:09.284259+0000","last_fullsized":"2024-09-10T18:36:09.284259+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:55:37.262746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.296014+0000","last_change":"2024-09-10T18:34:16.791941+0000","last_active":"2024-09-10T18:36:09.296014+0000","last_peered":"2024-09-10T18:36:09.296014+0000","last_clean":"2024-09-10T18:36:09.296014+0000","last_became_active":"2024-09-10T18:34:16.791642+0000","last_became_peered":"2024-09-10T18:34:16.791642+0000","last_unstale":"2024-09-10T18:36:09.296014+0000","last_undegraded":"2024-09-10T18:36:09.296014+0000","last_fullsized":"2024-09-10T18:36:09.296014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:17.142556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"60'19","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235774+0000","last_change":"2024-09-10T18:34:12.408804+0000","last_active":"2024-09-10T18:35:09.235774+0000","last_peered":"2024-09-10T18:35:09.235774+0000","last_clean":"2024-09-10T18:35:09.235774+0000","last_became_active":"2024-09-10T18:34:12.408451+0000","last_became_peered":"2024-09-10T18:34:12.408451+0000","last_unstale":"2024-09-10T18:35:09.235774+0000","last_undegraded":"2024-09-10T18:35:09.235774+0000","last_fullsized":"2024-09-10T18:35:09.235774+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:54.810974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292788+0000","last_change":"2024-09-10T18:34:10.385622+0000","last_active":"2024-09-10T18:36:09.292788+0000","last_peered":"2024-09-10T18:36:09.292788+0000","last_clean":"2024-09-10T18:36:09.292788+0000","last_became_active":"2024-09-10T18:34:10.385331+0000","last_became_peered":"2024-09-10T18:34:10.385331+0000","last_unstale":"2024-09-10T18:36:09.292788+0000","last_undegraded":"2024-09-10T18:36:09.292788+0000","last_fullsized":"2024-09-10T18:36:09.292788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.539572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"60'11","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.840464+0000","last_change":"2024-09-10T18:34:14.757917+0000","last_active":"2024-09-10T18:37:08.840464+0000","last_peered":"2024-09-10T18:37:08.840464+0000","last_clean":"2024-09-10T18:37:08.840464+0000","last_became_active":"2024-09-10T18:34:14.757537+0000","last_became_peered":"2024-09-10T18:34:14.757537+0000","last_unstale":"2024-09-10T18:37:08.840464+0000","last_undegraded":"2024-09-10T18:37:08.840464+0000","last_fullsized":"2024-09-10T18:37:08.840464+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:37.472990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305299+0000","last_change":"2024-09-10T18:34:16.785700+0000","last_active":"2024-09-10T18:36:09.305299+0000","last_peered":"2024-09-10T18:36:09.305299+0000","last_clean":"2024-09-10T18:36:09.305299+0000","last_became_active":"2024-09-10T18:34:16.784170+0000","last_became_peered":"2024-09-10T18:34:16.784170+0000","last_unstale":"2024-09-10T18:36:09.305299+0000","last_undegraded":"2024-09-10T18:36:09.305299+0000","last_fullsized":"2024-09-10T18:36:09.305299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:45.303063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304283+0000","last_change":"2024-09-10T18:34:12.425361+0000","last_active":"2024-09-10T18:36:09.304283+0000","last_peered":"2024-09-10T18:36:09.304283+0000","last_clean":"2024-09-10T18:36:09.304283+0000","last_became_active":"2024-09-10T18:34:12.425037+0000","last_became_peered":"2024-09-10T18:34:12.425037+0000","last_unstale":"2024-09-10T18:36:09.304283+0000","last_undegraded":"2024-09-10T18:36:09.304283+0000","last_fullsized":"2024-09-10T18:36:09.304283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:26.724110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.299600+0000","last_change":"2024-09-10T18:34:10.394017+0000","last_active":"2024-09-10T18:36:09.299600+0000","last_peered":"2024-09-10T18:36:09.299600+0000","last_clean":"2024-09-10T18:36:09.299600+0000","last_became_active":"2024-09-10T18:34:10.393675+0000","last_became_peered":"2024-09-10T18:34:10.393675+0000","last_unstale":"2024-09-10T18:36:09.299600+0000","last_undegraded":"2024-09-10T18:36:09.299600+0000","last_fullsized":"2024-09-10T18:36:09.299600+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.977012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284490+0000","last_change":"2024-09-10T18:34:14.754706+0000","last_active":"2024-09-10T18:36:09.284490+0000","last_peered":"2024-09-10T18:36:09.284490+0000","last_clean":"2024-09-10T18:36:09.284490+0000","last_became_active":"2024-09-10T18:34:14.754412+0000","last_became_peered":"2024-09-10T18:34:14.754412+0000","last_unstale":"2024-09-10T18:36:09.284490+0000","last_undegraded":"2024-09-10T18:36:09.284490+0000","last_fullsized":"2024-09-10T18:36:09.284490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:14.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285226+0000","last_change":"2024-09-10T18:34:16.773881+0000","last_active":"2024-09-10T18:36:09.285226+0000","last_peered":"2024-09-10T18:36:09.285226+0000","last_clean":"2024-09-10T18:36:09.285226+0000","last_became_active":"2024-09-10T18:34:16.773546+0000","last_became_peered":"2024-09-10T18:34:16.773546+0000","last_unstale":"2024-09-10T18:36:09.285226+0000","last_undegraded":"2024-09-10T18:36:09.285226+0000","last_fullsized":"2024-09-10T18:36:09.285226+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:01.758039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302501+0000","last_change":"2024-09-10T18:34:12.425588+0000","last_active":"2024-09-10T18:36:09.302501+0000","last_peered":"2024-09-10T18:36:09.302501+0000","last_clean":"2024-09-10T18:36:09.302501+0000","last_became_active":"2024-09-10T18:34:12.425168+0000","last_became_peered":"2024-09-10T18:34:12.425168+0000","last_unstale":"2024-09-10T18:36:09.302501+0000","last_undegraded":"2024-09-10T18:36:09.302501+0000","last_fullsized":"2024-09-10T18:36:09.302501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:22.016575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303716+0000","last_change":"2024-09-10T18:34:10.393121+0000","last_active":"2024-09-10T18:36:09.303716+0000","last_peered":"2024-09-10T18:36:09.303716+0000","last_clean":"2024-09-10T18:36:09.303716+0000","last_became_active":"2024-09-10T18:34:10.392805+0000","last_became_peered":"2024-09-10T18:34:10.392805+0000","last_unstale":"2024-09-10T18:36:09.303716+0000","last_undegraded":"2024-09-10T18:36:09.303716+0000","last_fullsized":"2024-09-10T18:36:09.303716+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:15:16.958863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304754+0000","last_change":"2024-09-10T18:34:14.766314+0000","last_active":"2024-09-10T18:36:09.304754+0000","last_peered":"2024-09-10T18:36:09.304754+0000","last_clean":"2024-09-10T18:36:09.304754+0000","last_became_active":"2024-09-10T18:34:14.765909+0000","last_became_peered":"2024-09-10T18:34:14.765909+0000","last_unstale":"2024-09-10T18:36:09.304754+0000","last_undegraded":"2024-09-10T18:36:09.304754+0000","last_fullsized":"2024-09-10T18:36:09.304754+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:32.963209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294414+0000","last_change":"2024-09-10T18:34:16.776881+0000","last_active":"2024-09-10T18:36:09.294414+0000","last_peered":"2024-09-10T18:36:09.294414+0000","last_clean":"2024-09-10T18:36:09.294414+0000","last_became_active":"2024-09-10T18:34:16.776413+0000","last_became_peered":"2024-09-10T18:34:16.776413+0000","last_unstale":"2024-09-10T18:36:09.294414+0000","last_undegraded":"2024-09-10T18:36:09.294414+0000","last_fullsized":"2024-09-10T18:36:09.294414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:59.503843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"60'12","reported_seq":44,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284487+0000","last_change":"2024-09-10T18:34:12.532970+0000","last_active":"2024-09-10T18:36:09.284487+0000","last_peered":"2024-09-10T18:36:09.284487+0000","last_clean":"2024-09-10T18:36:09.284487+0000","last_became_active":"2024-09-10T18:34:12.532083+0000","last_became_peered":"2024-09-10T18:34:12.532083+0000","last_unstale":"2024-09-10T18:36:09.284487+0000","last_undegraded":"2024-09-10T18:36:09.284487+0000","last_fullsized":"2024-09-10T18:36:09.284487+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:48.995415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291552+0000","last_change":"2024-09-10T18:34:10.386593+0000","last_active":"2024-09-10T18:36:09.291552+0000","last_peered":"2024-09-10T18:36:09.291552+0000","last_clean":"2024-09-10T18:36:09.291552+0000","last_became_active":"2024-09-10T18:34:10.386311+0000","last_became_peered":"2024-09-10T18:34:10.386311+0000","last_unstale":"2024-09-10T18:36:09.291552+0000","last_undegraded":"2024-09-10T18:36:09.291552+0000","last_fullsized":"2024-09-10T18:36:09.291552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:13.760929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.840434+0000","last_change":"2024-09-10T18:34:14.755395+0000","last_active":"2024-09-10T18:37:08.840434+0000","last_peered":"2024-09-10T18:37:08.840434+0000","last_clean":"2024-09-10T18:37:08.840434+0000","last_became_active":"2024-09-10T18:34:14.755096+0000","last_became_peered":"2024-09-10T18:34:14.755096+0000","last_unstale":"2024-09-10T18:37:08.840434+0000","last_undegraded":"2024-09-10T18:37:08.840434+0000","last_fullsized":"2024-09-10T18:37:08.840434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:24.068010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303027+0000","last_change":"2024-09-10T18:34:17.071114+0000","last_active":"2024-09-10T18:36:09.303027+0000","last_peered":"2024-09-10T18:36:09.303027+0000","last_clean":"2024-09-10T18:36:09.303027+0000","last_became_active":"2024-09-10T18:34:17.070207+0000","last_became_peered":"2024-09-10T18:34:17.070207+0000","last_unstale":"2024-09-10T18:36:09.303027+0000","last_undegraded":"2024-09-10T18:36:09.303027+0000","last_fullsized":"2024-09-10T18:36:09.303027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:53.923105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"60'12","reported_seq":39,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297432+0000","last_change":"2024-09-10T18:34:12.417008+0000","last_active":"2024-09-10T18:36:09.297432+0000","last_peered":"2024-09-10T18:36:09.297432+0000","last_clean":"2024-09-10T18:36:09.297432+0000","last_became_active":"2024-09-10T18:34:12.416463+0000","last_became_peered":"2024-09-10T18:34:12.416463+0000","last_unstale":"2024-09-10T18:36:09.297432+0000","last_undegraded":"2024-09-10T18:36:09.297432+0000","last_fullsized":"2024-09-10T18:36:09.297432+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:43.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.237443+0000","last_change":"2024-09-10T18:34:10.386812+0000","last_active":"2024-09-10T18:35:09.237443+0000","last_peered":"2024-09-10T18:35:09.237443+0000","last_clean":"2024-09-10T18:35:09.237443+0000","last_became_active":"2024-09-10T18:34:10.386530+0000","last_became_peered":"2024-09-10T18:34:10.386530+0000","last_unstale":"2024-09-10T18:35:09.237443+0000","last_undegraded":"2024-09-10T18:35:09.237443+0000","last_fullsized":"2024-09-10T18:35:09.237443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:23.383316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283765+0000","last_change":"2024-09-10T18:34:14.759668+0000","last_active":"2024-09-10T18:36:09.283765+0000","last_peered":"2024-09-10T18:36:09.283765+0000","last_clean":"2024-09-10T18:36:09.283765+0000","last_became_active":"2024-09-10T18:34:14.759403+0000","last_became_peered":"2024-09-10T18:34:14.759403+0000","last_unstale":"2024-09-10T18:36:09.283765+0000","last_undegraded":"2024-09-10T18:36:09.283765+0000","last_fullsized":"2024-09-10T18:36:09.283765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:10.185572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"64'5","reported_seq":185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:14.173776+0000","last_change":"2024-09-10T18:34:26.912914+0000","last_active":"2024-09-10T18:37:14.173776+0000","last_peered":"2024-09-10T18:37:14.173776+0000","last_clean":"2024-09-10T18:37:14.173776+0000","last_became_active":"2024-09-10T18:34:15.776286+0000","last_became_peered":"2024-09-10T18:34:15.776286+0000","last_unstale":"2024-09-10T18:37:14.173776+0000","last_undegraded":"2024-09-10T18:37:14.173776+0000","last_fullsized":"2024-09-10T18:37:14.173776+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_clean_scrub_stamp":"2024-09-10T18:34:14.742092+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:10.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014544790000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":159,"num_read_kb":154,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292347+0000","last_change":"2024-09-10T18:34:17.060699+0000","last_active":"2024-09-10T18:36:09.292347+0000","last_peered":"2024-09-10T18:36:09.292347+0000","last_clean":"2024-09-10T18:36:09.292347+0000","last_became_active":"2024-09-10T18:34:17.059974+0000","last_became_peered":"2024-09-10T18:34:17.059974+0000","last_unstale":"2024-09-10T18:36:09.292347+0000","last_undegraded":"2024-09-10T18:36:09.292347+0000","last_fullsized":"2024-09-10T18:36:09.292347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:22.982136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"60'13","reported_seq":48,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305015+0000","last_change":"2024-09-10T18:34:12.425358+0000","last_active":"2024-09-10T18:36:09.305015+0000","last_peered":"2024-09-10T18:36:09.305015+0000","last_clean":"2024-09-10T18:36:09.305015+0000","last_became_active":"2024-09-10T18:34:12.425014+0000","last_became_peered":"2024-09-10T18:34:12.425014+0000","last_unstale":"2024-09-10T18:36:09.305015+0000","last_undegraded":"2024-09-10T18:36:09.305015+0000","last_fullsized":"2024-09-10T18:36:09.305015+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:32.919803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"53'1","reported_seq":32,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304651+0000","last_change":"2024-09-10T18:34:10.388652+0000","last_active":"2024-09-10T18:36:09.304651+0000","last_peered":"2024-09-10T18:36:09.304651+0000","last_clean":"2024-09-10T18:36:09.304651+0000","last_became_active":"2024-09-10T18:34:10.388262+0000","last_became_peered":"2024-09-10T18:34:10.388262+0000","last_unstale":"2024-09-10T18:36:09.304651+0000","last_undegraded":"2024-09-10T18:36:09.304651+0000","last_fullsized":"2024-09-10T18:36:09.304651+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:49:55.976160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"60'11","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.854928+0000","last_change":"2024-09-10T18:34:14.757815+0000","last_active":"2024-09-10T18:37:08.854928+0000","last_peered":"2024-09-10T18:37:08.854928+0000","last_clean":"2024-09-10T18:37:08.854928+0000","last_became_active":"2024-09-10T18:34:14.757469+0000","last_became_peered":"2024-09-10T18:34:14.757469+0000","last_unstale":"2024-09-10T18:37:08.854928+0000","last_undegraded":"2024-09-10T18:37:08.854928+0000","last_fullsized":"2024-09-10T18:37:08.854928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:11.124270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"61'1","reported_seq":24,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.283719+0000","last_change":"2024-09-10T18:34:26.917522+0000","last_active":"2024-09-10T18:36:09.283719+0000","last_peered":"2024-09-10T18:36:09.283719+0000","last_clean":"2024-09-10T18:36:09.283719+0000","last_became_active":"2024-09-10T18:34:15.761086+0000","last_became_peered":"2024-09-10T18:34:15.761086+0000","last_unstale":"2024-09-10T18:36:09.283719+0000","last_undegraded":"2024-09-10T18:36:09.283719+0000","last_fullsized":"2024-09-10T18:36:09.283719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_clean_scrub_stamp":"2024-09-10T18:34:14.742092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:24.966817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086406500000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284148+0000","last_change":"2024-09-10T18:34:16.768373+0000","last_active":"2024-09-10T18:36:09.284148+0000","last_peered":"2024-09-10T18:36:09.284148+0000","last_clean":"2024-09-10T18:36:09.284148+0000","last_became_active":"2024-09-10T18:34:16.768067+0000","last_became_peered":"2024-09-10T18:34:16.768067+0000","last_unstale":"2024-09-10T18:36:09.284148+0000","last_undegraded":"2024-09-10T18:36:09.284148+0000","last_fullsized":"2024-09-10T18:36:09.284148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:25.947678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"60'30","reported_seq":108,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.854557+0000","last_change":"2024-09-10T18:34:12.544095+0000","last_active":"2024-09-10T18:37:08.854557+0000","last_peered":"2024-09-10T18:37:08.854557+0000","last_clean":"2024-09-10T18:37:08.854557+0000","last_became_active":"2024-09-10T18:34:12.543696+0000","last_became_peered":"2024-09-10T18:34:12.543696+0000","last_unstale":"2024-09-10T18:37:08.854557+0000","last_undegraded":"2024-09-10T18:37:08.854557+0000","last_fullsized":"2024-09-10T18:37:08.854557+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:59.615050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293198+0000","last_change":"2024-09-10T18:34:10.389037+0000","last_active":"2024-09-10T18:36:09.293198+0000","last_peered":"2024-09-10T18:36:09.293198+0000","last_clean":"2024-09-10T18:36:09.293198+0000","last_became_active":"2024-09-10T18:34:10.388734+0000","last_became_peered":"2024-09-10T18:34:10.388734+0000","last_unstale":"2024-09-10T18:36:09.293198+0000","last_undegraded":"2024-09-10T18:36:09.293198+0000","last_fullsized":"2024-09-10T18:36:09.293198+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:40.596707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"60'11","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.854576+0000","last_change":"2024-09-10T18:34:14.765846+0000","last_active":"2024-09-10T18:37:08.854576+0000","last_peered":"2024-09-10T18:37:08.854576+0000","last_clean":"2024-09-10T18:37:08.854576+0000","last_became_active":"2024-09-10T18:34:14.765312+0000","last_became_peered":"2024-09-10T18:34:14.765312+0000","last_unstale":"2024-09-10T18:37:08.854576+0000","last_undegraded":"2024-09-10T18:37:08.854576+0000","last_fullsized":"2024-09-10T18:37:08.854576+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:20.012037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"63'2","reported_seq":24,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235156+0000","last_change":"2024-09-10T18:34:26.907339+0000","last_active":"2024-09-10T18:35:09.235156+0000","last_peered":"2024-09-10T18:35:09.235156+0000","last_clean":"2024-09-10T18:35:09.235156+0000","last_became_active":"2024-09-10T18:34:15.760794+0000","last_became_peered":"2024-09-10T18:34:15.760794+0000","last_unstale":"2024-09-10T18:35:09.235156+0000","last_undegraded":"2024-09-10T18:35:09.235156+0000","last_fullsized":"2024-09-10T18:35:09.235156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:14.742092+0000","last_clean_scrub_stamp":"2024-09-10T18:34:14.742092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:58.281474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001795804,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303435+0000","last_change":"2024-09-10T18:34:17.071309+0000","last_active":"2024-09-10T18:36:09.303435+0000","last_peered":"2024-09-10T18:36:09.303435+0000","last_clean":"2024-09-10T18:36:09.303435+0000","last_became_active":"2024-09-10T18:34:17.070392+0000","last_became_peered":"2024-09-10T18:34:17.070392+0000","last_unstale":"2024-09-10T18:36:09.303435+0000","last_undegraded":"2024-09-10T18:36:09.303435+0000","last_fullsized":"2024-09-10T18:36:09.303435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:50.193185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"60'14","reported_seq":43,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294796+0000","last_change":"2024-09-10T18:34:12.528911+0000","last_active":"2024-09-10T18:36:09.294796+0000","last_peered":"2024-09-10T18:36:09.294796+0000","last_clean":"2024-09-10T18:36:09.294796+0000","last_became_active":"2024-09-10T18:34:12.528095+0000","last_became_peered":"2024-09-10T18:34:12.528095+0000","last_unstale":"2024-09-10T18:36:09.294796+0000","last_undegraded":"2024-09-10T18:36:09.294796+0000","last_fullsized":"2024-09-10T18:36:09.294796+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:30.917283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.311047+0000","last_change":"2024-09-10T18:36:10.311047+0000","last_active":"2024-09-10T18:36:10.311047+0000","last_peered":"2024-09-10T18:36:10.311047+0000","last_clean":"2024-09-10T18:36:10.311047+0000","last_became_active":"2024-09-10T18:36:10.310735+0000","last_became_peered":"2024-09-10T18:36:10.310735+0000","last_unstale":"2024-09-10T18:36:10.311047+0000","last_undegraded":"2024-09-10T18:36:10.311047+0000","last_fullsized":"2024-09-10T18:36:10.311047+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:27.207694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298932+0000","last_change":"2024-09-10T18:34:14.770112+0000","last_active":"2024-09-10T18:36:09.298932+0000","last_peered":"2024-09-10T18:36:09.298932+0000","last_clean":"2024-09-10T18:36:09.298932+0000","last_became_active":"2024-09-10T18:34:14.769800+0000","last_became_peered":"2024-09-10T18:34:14.769800+0000","last_unstale":"2024-09-10T18:36:09.298932+0000","last_undegraded":"2024-09-10T18:36:09.298932+0000","last_fullsized":"2024-09-10T18:36:09.298932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:28.581637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.296280+0000","last_change":"2024-09-10T18:34:16.788769+0000","last_active":"2024-09-10T18:36:09.296280+0000","last_peered":"2024-09-10T18:36:09.296280+0000","last_clean":"2024-09-10T18:36:09.296280+0000","last_became_active":"2024-09-10T18:34:16.788476+0000","last_became_peered":"2024-09-10T18:34:16.788476+0000","last_unstale":"2024-09-10T18:36:09.296280+0000","last_undegraded":"2024-09-10T18:36:09.296280+0000","last_fullsized":"2024-09-10T18:36:09.296280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:28.124907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"60'19","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.286040+0000","last_change":"2024-09-10T18:34:12.410505+0000","last_active":"2024-09-10T18:36:09.286040+0000","last_peered":"2024-09-10T18:36:09.286040+0000","last_clean":"2024-09-10T18:36:09.286040+0000","last_became_active":"2024-09-10T18:34:12.410091+0000","last_became_peered":"2024-09-10T18:34:12.410091+0000","last_unstale":"2024-09-10T18:36:09.286040+0000","last_undegraded":"2024-09-10T18:36:09.286040+0000","last_fullsized":"2024-09-10T18:36:09.286040+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:27.929172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294709+0000","last_change":"2024-09-10T18:34:10.381107+0000","last_active":"2024-09-10T18:36:09.294709+0000","last_peered":"2024-09-10T18:36:09.294709+0000","last_clean":"2024-09-10T18:36:09.294709+0000","last_became_active":"2024-09-10T18:34:10.380610+0000","last_became_peered":"2024-09-10T18:34:10.380610+0000","last_unstale":"2024-09-10T18:36:09.294709+0000","last_undegraded":"2024-09-10T18:36:09.294709+0000","last_fullsized":"2024-09-10T18:36:09.294709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:55:46.895780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.300231+0000","last_change":"2024-09-10T18:34:14.762478+0000","last_active":"2024-09-10T18:36:09.300231+0000","last_peered":"2024-09-10T18:36:09.300231+0000","last_clean":"2024-09-10T18:36:09.300231+0000","last_became_active":"2024-09-10T18:34:14.761726+0000","last_became_peered":"2024-09-10T18:34:14.761726+0000","last_unstale":"2024-09-10T18:36:09.300231+0000","last_undegraded":"2024-09-10T18:36:09.300231+0000","last_fullsized":"2024-09-10T18:36:09.300231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:44.927582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303953+0000","last_change":"2024-09-10T18:34:16.786378+0000","last_active":"2024-09-10T18:36:09.303953+0000","last_peered":"2024-09-10T18:36:09.303953+0000","last_clean":"2024-09-10T18:36:09.303953+0000","last_became_active":"2024-09-10T18:34:16.786077+0000","last_became_peered":"2024-09-10T18:34:16.786077+0000","last_unstale":"2024-09-10T18:36:09.303953+0000","last_undegraded":"2024-09-10T18:36:09.303953+0000","last_fullsized":"2024-09-10T18:36:09.303953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:24.172589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"60'18","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.295939+0000","last_change":"2024-09-10T18:34:12.540632+0000","last_active":"2024-09-10T18:36:09.295939+0000","last_peered":"2024-09-10T18:36:09.295939+0000","last_clean":"2024-09-10T18:36:09.295939+0000","last_became_active":"2024-09-10T18:34:12.540348+0000","last_became_peered":"2024-09-10T18:34:12.540348+0000","last_unstale":"2024-09-10T18:36:09.295939+0000","last_undegraded":"2024-09-10T18:36:09.295939+0000","last_fullsized":"2024-09-10T18:36:09.295939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:51.309938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305462+0000","last_change":"2024-09-10T18:34:10.397178+0000","last_active":"2024-09-10T18:36:09.305462+0000","last_peered":"2024-09-10T18:36:09.305462+0000","last_clean":"2024-09-10T18:36:09.305462+0000","last_became_active":"2024-09-10T18:34:10.396865+0000","last_became_peered":"2024-09-10T18:34:10.396865+0000","last_unstale":"2024-09-10T18:36:09.305462+0000","last_undegraded":"2024-09-10T18:36:09.305462+0000","last_fullsized":"2024-09-10T18:36:09.305462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:42.452022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.663025+0000","last_change":"2024-09-10T18:36:10.663025+0000","last_active":"2024-09-10T18:36:10.663025+0000","last_peered":"2024-09-10T18:36:10.663025+0000","last_clean":"2024-09-10T18:36:10.663025+0000","last_became_active":"2024-09-10T18:36:10.662169+0000","last_became_peered":"2024-09-10T18:36:10.662169+0000","last_unstale":"2024-09-10T18:36:10.663025+0000","last_undegraded":"2024-09-10T18:36:10.663025+0000","last_fullsized":"2024-09-10T18:36:10.663025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:43.803320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292694+0000","last_change":"2024-09-10T18:34:17.062882+0000","last_active":"2024-09-10T18:36:09.292694+0000","last_peered":"2024-09-10T18:36:09.292694+0000","last_clean":"2024-09-10T18:36:09.292694+0000","last_became_active":"2024-09-10T18:34:17.062544+0000","last_became_peered":"2024-09-10T18:34:17.062544+0000","last_unstale":"2024-09-10T18:36:09.292694+0000","last_undegraded":"2024-09-10T18:36:09.292694+0000","last_fullsized":"2024-09-10T18:36:09.292694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:12.227256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"60'14","reported_seq":42,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298912+0000","last_change":"2024-09-10T18:34:12.419858+0000","last_active":"2024-09-10T18:36:09.298912+0000","last_peered":"2024-09-10T18:36:09.298912+0000","last_clean":"2024-09-10T18:36:09.298912+0000","last_became_active":"2024-09-10T18:34:12.419525+0000","last_became_peered":"2024-09-10T18:34:12.419525+0000","last_unstale":"2024-09-10T18:36:09.298912+0000","last_undegraded":"2024-09-10T18:36:09.298912+0000","last_fullsized":"2024-09-10T18:36:09.298912+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:56.699819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291680+0000","last_change":"2024-09-10T18:34:10.386886+0000","last_active":"2024-09-10T18:36:09.291680+0000","last_peered":"2024-09-10T18:36:09.291680+0000","last_clean":"2024-09-10T18:36:09.291680+0000","last_became_active":"2024-09-10T18:34:10.386597+0000","last_became_peered":"2024-09-10T18:34:10.386597+0000","last_unstale":"2024-09-10T18:36:09.291680+0000","last_undegraded":"2024-09-10T18:36:09.291680+0000","last_fullsized":"2024-09-10T18:36:09.291680+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:05.511409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297564+0000","last_change":"2024-09-10T18:34:14.760467+0000","last_active":"2024-09-10T18:36:09.297564+0000","last_peered":"2024-09-10T18:36:09.297564+0000","last_clean":"2024-09-10T18:36:09.297564+0000","last_became_active":"2024-09-10T18:34:14.760125+0000","last_became_peered":"2024-09-10T18:34:14.760125+0000","last_unstale":"2024-09-10T18:36:09.297564+0000","last_undegraded":"2024-09-10T18:36:09.297564+0000","last_fullsized":"2024-09-10T18:36:09.297564+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:49.349310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.300108+0000","last_change":"2024-09-10T18:34:16.788735+0000","last_active":"2024-09-10T18:36:09.300108+0000","last_peered":"2024-09-10T18:36:09.300108+0000","last_clean":"2024-09-10T18:36:09.300108+0000","last_became_active":"2024-09-10T18:34:16.788438+0000","last_became_peered":"2024-09-10T18:34:16.788438+0000","last_unstale":"2024-09-10T18:36:09.300108+0000","last_undegraded":"2024-09-10T18:36:09.300108+0000","last_fullsized":"2024-09-10T18:36:09.300108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:18.081970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302731+0000","last_change":"2024-09-10T18:34:12.426576+0000","last_active":"2024-09-10T18:36:09.302731+0000","last_peered":"2024-09-10T18:36:09.302731+0000","last_clean":"2024-09-10T18:36:09.302731+0000","last_became_active":"2024-09-10T18:34:12.426080+0000","last_became_peered":"2024-09-10T18:34:12.426080+0000","last_unstale":"2024-09-10T18:36:09.302731+0000","last_undegraded":"2024-09-10T18:36:09.302731+0000","last_fullsized":"2024-09-10T18:36:09.302731+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:47.530597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294381+0000","last_change":"2024-09-10T18:34:10.378622+0000","last_active":"2024-09-10T18:36:09.294381+0000","last_peered":"2024-09-10T18:36:09.294381+0000","last_clean":"2024-09-10T18:36:09.294381+0000","last_became_active":"2024-09-10T18:34:10.378395+0000","last_became_peered":"2024-09-10T18:34:10.378395+0000","last_unstale":"2024-09-10T18:36:09.294381+0000","last_undegraded":"2024-09-10T18:36:09.294381+0000","last_fullsized":"2024-09-10T18:36:09.294381+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:39.264870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"65'79","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291877+0000","last_change":"2024-09-10T18:33:28.631139+0000","last_active":"2024-09-10T18:36:09.291877+0000","last_peered":"2024-09-10T18:36:09.291877+0000","last_clean":"2024-09-10T18:36:09.291877+0000","last_became_active":"2024-09-10T18:33:28.620160+0000","last_became_peered":"2024-09-10T18:33:28.620160+0000","last_unstale":"2024-09-10T18:36:09.291877+0000","last_undegraded":"2024-09-10T18:36:09.291877+0000","last_fullsized":"2024-09-10T18:36:09.291877+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:28:49.953203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:28:49.953203+0000","last_clean_scrub_stamp":"2024-09-10T18:28:49.953203+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:01.528284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.236361+0000","last_change":"2024-09-10T18:34:14.760355+0000","last_active":"2024-09-10T18:35:09.236361+0000","last_peered":"2024-09-10T18:35:09.236361+0000","last_clean":"2024-09-10T18:35:09.236361+0000","last_became_active":"2024-09-10T18:34:14.760023+0000","last_became_peered":"2024-09-10T18:34:14.760023+0000","last_unstale":"2024-09-10T18:35:09.236361+0000","last_undegraded":"2024-09-10T18:35:09.236361+0000","last_fullsized":"2024-09-10T18:35:09.236361+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:54.765145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294691+0000","last_change":"2024-09-10T18:34:16.775535+0000","last_active":"2024-09-10T18:36:09.294691+0000","last_peered":"2024-09-10T18:36:09.294691+0000","last_clean":"2024-09-10T18:36:09.294691+0000","last_became_active":"2024-09-10T18:34:16.775183+0000","last_became_peered":"2024-09-10T18:34:16.775183+0000","last_unstale":"2024-09-10T18:36:09.294691+0000","last_undegraded":"2024-09-10T18:36:09.294691+0000","last_fullsized":"2024-09-10T18:36:09.294691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:25.153121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"60'17","reported_seq":49,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293071+0000","last_change":"2024-09-10T18:34:12.412677+0000","last_active":"2024-09-10T18:36:09.293071+0000","last_peered":"2024-09-10T18:36:09.293071+0000","last_clean":"2024-09-10T18:36:09.293071+0000","last_became_active":"2024-09-10T18:34:12.412384+0000","last_became_peered":"2024-09-10T18:34:12.412384+0000","last_unstale":"2024-09-10T18:36:09.293071+0000","last_undegraded":"2024-09-10T18:36:09.293071+0000","last_fullsized":"2024-09-10T18:36:09.293071+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:41.318982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303991+0000","last_change":"2024-09-10T18:34:10.390836+0000","last_active":"2024-09-10T18:36:09.303991+0000","last_peered":"2024-09-10T18:36:09.303991+0000","last_clean":"2024-09-10T18:36:09.303991+0000","last_became_active":"2024-09-10T18:34:10.390162+0000","last_became_peered":"2024-09-10T18:34:10.390162+0000","last_unstale":"2024-09-10T18:36:09.303991+0000","last_undegraded":"2024-09-10T18:36:09.303991+0000","last_fullsized":"2024-09-10T18:36:09.303991+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:38.705903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235346+0000","last_change":"2024-09-10T18:34:14.753040+0000","last_active":"2024-09-10T18:35:09.235346+0000","last_peered":"2024-09-10T18:35:09.235346+0000","last_clean":"2024-09-10T18:35:09.235346+0000","last_became_active":"2024-09-10T18:34:14.752804+0000","last_became_peered":"2024-09-10T18:34:14.752804+0000","last_unstale":"2024-09-10T18:35:09.235346+0000","last_undegraded":"2024-09-10T18:35:09.235346+0000","last_fullsized":"2024-09-10T18:35:09.235346+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:41.654922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.291774+0000","last_change":"2024-09-10T18:34:16.779778+0000","last_active":"2024-09-10T18:36:09.291774+0000","last_peered":"2024-09-10T18:36:09.291774+0000","last_clean":"2024-09-10T18:36:09.291774+0000","last_became_active":"2024-09-10T18:34:16.779441+0000","last_became_peered":"2024-09-10T18:34:16.779441+0000","last_unstale":"2024-09-10T18:36:09.291774+0000","last_undegraded":"2024-09-10T18:36:09.291774+0000","last_fullsized":"2024-09-10T18:36:09.291774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:51.472906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294242+0000","last_change":"2024-09-10T18:34:12.414377+0000","last_active":"2024-09-10T18:36:09.294242+0000","last_peered":"2024-09-10T18:36:09.294242+0000","last_clean":"2024-09-10T18:36:09.294242+0000","last_became_active":"2024-09-10T18:34:12.413793+0000","last_became_peered":"2024-09-10T18:34:12.413793+0000","last_unstale":"2024-09-10T18:36:09.294242+0000","last_undegraded":"2024-09-10T18:36:09.294242+0000","last_fullsized":"2024-09-10T18:36:09.294242+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:21.235880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301544+0000","last_change":"2024-09-10T18:34:10.384603+0000","last_active":"2024-09-10T18:36:09.301544+0000","last_peered":"2024-09-10T18:36:09.301544+0000","last_clean":"2024-09-10T18:36:09.301544+0000","last_became_active":"2024-09-10T18:34:10.384238+0000","last_became_peered":"2024-09-10T18:34:10.384238+0000","last_unstale":"2024-09-10T18:36:09.301544+0000","last_undegraded":"2024-09-10T18:36:09.301544+0000","last_fullsized":"2024-09-10T18:36:09.301544+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:37.702399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297804+0000","last_change":"2024-09-10T18:34:14.773539+0000","last_active":"2024-09-10T18:36:09.297804+0000","last_peered":"2024-09-10T18:36:09.297804+0000","last_clean":"2024-09-10T18:36:09.297804+0000","last_became_active":"2024-09-10T18:34:14.773211+0000","last_became_peered":"2024-09-10T18:34:14.773211+0000","last_unstale":"2024-09-10T18:36:09.297804+0000","last_undegraded":"2024-09-10T18:36:09.297804+0000","last_fullsized":"2024-09-10T18:36:09.297804+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:59.702492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298775+0000","last_change":"2024-09-10T18:34:16.790464+0000","last_active":"2024-09-10T18:36:09.298775+0000","last_peered":"2024-09-10T18:36:09.298775+0000","last_clean":"2024-09-10T18:36:09.298775+0000","last_became_active":"2024-09-10T18:34:16.790053+0000","last_became_peered":"2024-09-10T18:34:16.790053+0000","last_unstale":"2024-09-10T18:36:09.298775+0000","last_undegraded":"2024-09-10T18:36:09.298775+0000","last_fullsized":"2024-09-10T18:36:09.298775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:39.152799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292948+0000","last_change":"2024-09-10T18:34:12.415136+0000","last_active":"2024-09-10T18:36:09.292948+0000","last_peered":"2024-09-10T18:36:09.292948+0000","last_clean":"2024-09-10T18:36:09.292948+0000","last_became_active":"2024-09-10T18:34:12.414635+0000","last_became_peered":"2024-09-10T18:34:12.414635+0000","last_unstale":"2024-09-10T18:36:09.292948+0000","last_undegraded":"2024-09-10T18:36:09.292948+0000","last_fullsized":"2024-09-10T18:36:09.292948+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:57.182105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302353+0000","last_change":"2024-09-10T18:34:10.393336+0000","last_active":"2024-09-10T18:36:09.302353+0000","last_peered":"2024-09-10T18:36:09.302353+0000","last_clean":"2024-09-10T18:36:09.302353+0000","last_became_active":"2024-09-10T18:34:10.392912+0000","last_became_peered":"2024-09-10T18:34:10.392912+0000","last_unstale":"2024-09-10T18:36:09.302353+0000","last_undegraded":"2024-09-10T18:36:09.302353+0000","last_fullsized":"2024-09-10T18:36:09.302353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:51.401558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293676+0000","last_change":"2024-09-10T18:34:14.752413+0000","last_active":"2024-09-10T18:36:09.293676+0000","last_peered":"2024-09-10T18:36:09.293676+0000","last_clean":"2024-09-10T18:36:09.293676+0000","last_became_active":"2024-09-10T18:34:14.751793+0000","last_became_peered":"2024-09-10T18:34:14.751793+0000","last_unstale":"2024-09-10T18:36:09.293676+0000","last_undegraded":"2024-09-10T18:36:09.293676+0000","last_fullsized":"2024-09-10T18:36:09.293676+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:18.884416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294234+0000","last_change":"2024-09-10T18:34:17.057325+0000","last_active":"2024-09-10T18:36:09.294234+0000","last_peered":"2024-09-10T18:36:09.294234+0000","last_clean":"2024-09-10T18:36:09.294234+0000","last_became_active":"2024-09-10T18:34:17.056572+0000","last_became_peered":"2024-09-10T18:34:17.056572+0000","last_unstale":"2024-09-10T18:36:09.294234+0000","last_undegraded":"2024-09-10T18:36:09.294234+0000","last_fullsized":"2024-09-10T18:36:09.294234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:43.185973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293442+0000","last_change":"2024-09-10T18:34:12.415162+0000","last_active":"2024-09-10T18:36:09.293442+0000","last_peered":"2024-09-10T18:36:09.293442+0000","last_clean":"2024-09-10T18:36:09.293442+0000","last_became_active":"2024-09-10T18:34:12.414822+0000","last_became_peered":"2024-09-10T18:34:12.414822+0000","last_unstale":"2024-09-10T18:36:09.293442+0000","last_undegraded":"2024-09-10T18:36:09.293442+0000","last_fullsized":"2024-09-10T18:36:09.293442+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:56.490961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285490+0000","last_change":"2024-09-10T18:34:10.384963+0000","last_active":"2024-09-10T18:36:09.285490+0000","last_peered":"2024-09-10T18:36:09.285490+0000","last_clean":"2024-09-10T18:36:09.285490+0000","last_became_active":"2024-09-10T18:34:10.384662+0000","last_became_peered":"2024-09-10T18:34:10.384662+0000","last_unstale":"2024-09-10T18:36:09.285490+0000","last_undegraded":"2024-09-10T18:36:09.285490+0000","last_fullsized":"2024-09-10T18:36:09.285490+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:47.505769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"60'11","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.840959+0000","last_change":"2024-09-10T18:34:14.758937+0000","last_active":"2024-09-10T18:37:08.840959+0000","last_peered":"2024-09-10T18:37:08.840959+0000","last_clean":"2024-09-10T18:37:08.840959+0000","last_became_active":"2024-09-10T18:34:14.758572+0000","last_became_peered":"2024-09-10T18:34:14.758572+0000","last_unstale":"2024-09-10T18:37:08.840959+0000","last_undegraded":"2024-09-10T18:37:08.840959+0000","last_fullsized":"2024-09-10T18:37:08.840959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:34.905779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304306+0000","last_change":"2024-09-10T18:34:16.786561+0000","last_active":"2024-09-10T18:36:09.304306+0000","last_peered":"2024-09-10T18:36:09.304306+0000","last_clean":"2024-09-10T18:36:09.304306+0000","last_became_active":"2024-09-10T18:34:16.786264+0000","last_became_peered":"2024-09-10T18:34:16.786264+0000","last_unstale":"2024-09-10T18:36:09.304306+0000","last_undegraded":"2024-09-10T18:36:09.304306+0000","last_fullsized":"2024-09-10T18:36:09.304306+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:41.116785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292189+0000","last_change":"2024-09-10T18:34:12.414409+0000","last_active":"2024-09-10T18:36:09.292189+0000","last_peered":"2024-09-10T18:36:09.292189+0000","last_clean":"2024-09-10T18:36:09.292189+0000","last_became_active":"2024-09-10T18:34:12.413996+0000","last_became_peered":"2024-09-10T18:34:12.413996+0000","last_unstale":"2024-09-10T18:36:09.292189+0000","last_undegraded":"2024-09-10T18:36:09.292189+0000","last_fullsized":"2024-09-10T18:36:09.292189+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:10.686059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304256+0000","last_change":"2024-09-10T18:34:10.390412+0000","last_active":"2024-09-10T18:36:09.304256+0000","last_peered":"2024-09-10T18:36:09.304256+0000","last_clean":"2024-09-10T18:36:09.304256+0000","last_became_active":"2024-09-10T18:34:10.390061+0000","last_became_peered":"2024-09-10T18:34:10.390061+0000","last_unstale":"2024-09-10T18:36:09.304256+0000","last_undegraded":"2024-09-10T18:36:09.304256+0000","last_fullsized":"2024-09-10T18:36:09.304256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:59.161343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297270+0000","last_change":"2024-09-10T18:34:14.759446+0000","last_active":"2024-09-10T18:36:09.297270+0000","last_peered":"2024-09-10T18:36:09.297270+0000","last_clean":"2024-09-10T18:36:09.297270+0000","last_became_active":"2024-09-10T18:34:14.758968+0000","last_became_peered":"2024-09-10T18:34:14.758968+0000","last_unstale":"2024-09-10T18:36:09.297270+0000","last_undegraded":"2024-09-10T18:36:09.297270+0000","last_fullsized":"2024-09-10T18:36:09.297270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:39.311406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302579+0000","last_change":"2024-09-10T18:34:16.785485+0000","last_active":"2024-09-10T18:36:09.302579+0000","last_peered":"2024-09-10T18:36:09.302579+0000","last_clean":"2024-09-10T18:36:09.302579+0000","last_became_active":"2024-09-10T18:34:16.785128+0000","last_became_peered":"2024-09-10T18:34:16.785128+0000","last_unstale":"2024-09-10T18:36:09.302579+0000","last_undegraded":"2024-09-10T18:36:09.302579+0000","last_fullsized":"2024-09-10T18:36:09.302579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:43.503900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"60'4","reported_seq":26,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.233698+0000","last_change":"2024-09-10T18:34:12.415419+0000","last_active":"2024-09-10T18:35:09.233698+0000","last_peered":"2024-09-10T18:35:09.233698+0000","last_clean":"2024-09-10T18:35:09.233698+0000","last_became_active":"2024-09-10T18:34:12.415127+0000","last_became_peered":"2024-09-10T18:34:12.415127+0000","last_unstale":"2024-09-10T18:35:09.233698+0000","last_undegraded":"2024-09-10T18:35:09.233698+0000","last_fullsized":"2024-09-10T18:35:09.233698+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:17.177881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.234818+0000","last_change":"2024-09-10T18:34:10.377274+0000","last_active":"2024-09-10T18:35:09.234818+0000","last_peered":"2024-09-10T18:35:09.234818+0000","last_clean":"2024-09-10T18:35:09.234818+0000","last_became_active":"2024-09-10T18:34:10.376958+0000","last_became_peered":"2024-09-10T18:34:10.376958+0000","last_unstale":"2024-09-10T18:35:09.234818+0000","last_undegraded":"2024-09-10T18:35:09.234818+0000","last_fullsized":"2024-09-10T18:35:09.234818+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:30.048373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304971+0000","last_change":"2024-09-10T18:34:14.761136+0000","last_active":"2024-09-10T18:36:09.304971+0000","last_peered":"2024-09-10T18:36:09.304971+0000","last_clean":"2024-09-10T18:36:09.304971+0000","last_became_active":"2024-09-10T18:34:14.760817+0000","last_became_peered":"2024-09-10T18:34:14.760817+0000","last_unstale":"2024-09-10T18:36:09.304971+0000","last_undegraded":"2024-09-10T18:36:09.304971+0000","last_fullsized":"2024-09-10T18:36:09.304971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:19.634840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292575+0000","last_change":"2024-09-10T18:34:17.062445+0000","last_active":"2024-09-10T18:36:09.292575+0000","last_peered":"2024-09-10T18:36:09.292575+0000","last_clean":"2024-09-10T18:36:09.292575+0000","last_became_active":"2024-09-10T18:34:17.061610+0000","last_became_peered":"2024-09-10T18:34:17.061610+0000","last_unstale":"2024-09-10T18:36:09.292575+0000","last_undegraded":"2024-09-10T18:36:09.292575+0000","last_fullsized":"2024-09-10T18:36:09.292575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:42.160284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"60'11","reported_seq":40,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292571+0000","last_change":"2024-09-10T18:34:12.527974+0000","last_active":"2024-09-10T18:36:09.292571+0000","last_peered":"2024-09-10T18:36:09.292571+0000","last_clean":"2024-09-10T18:36:09.292571+0000","last_became_active":"2024-09-10T18:34:12.526963+0000","last_became_peered":"2024-09-10T18:34:12.526963+0000","last_unstale":"2024-09-10T18:36:09.292571+0000","last_undegraded":"2024-09-10T18:36:09.292571+0000","last_fullsized":"2024-09-10T18:36:09.292571+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:20.824387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294627+0000","last_change":"2024-09-10T18:34:10.383808+0000","last_active":"2024-09-10T18:36:09.294627+0000","last_peered":"2024-09-10T18:36:09.294627+0000","last_clean":"2024-09-10T18:36:09.294627+0000","last_became_active":"2024-09-10T18:34:10.383378+0000","last_became_peered":"2024-09-10T18:34:10.383378+0000","last_unstale":"2024-09-10T18:36:09.294627+0000","last_undegraded":"2024-09-10T18:36:09.294627+0000","last_fullsized":"2024-09-10T18:36:09.294627+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:54.958532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"60'11","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.854765+0000","last_change":"2024-09-10T18:34:14.760088+0000","last_active":"2024-09-10T18:37:08.854765+0000","last_peered":"2024-09-10T18:37:08.854765+0000","last_clean":"2024-09-10T18:37:08.854765+0000","last_became_active":"2024-09-10T18:34:14.759776+0000","last_became_peered":"2024-09-10T18:34:14.759776+0000","last_unstale":"2024-09-10T18:37:08.854765+0000","last_undegraded":"2024-09-10T18:37:08.854765+0000","last_fullsized":"2024-09-10T18:37:08.854765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:08.637410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297014+0000","last_change":"2024-09-10T18:34:16.789419+0000","last_active":"2024-09-10T18:36:09.297014+0000","last_peered":"2024-09-10T18:36:09.297014+0000","last_clean":"2024-09-10T18:36:09.297014+0000","last_became_active":"2024-09-10T18:34:16.789098+0000","last_became_peered":"2024-09-10T18:34:16.789098+0000","last_unstale":"2024-09-10T18:36:09.297014+0000","last_undegraded":"2024-09-10T18:36:09.297014+0000","last_fullsized":"2024-09-10T18:36:09.297014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:04.952267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297111+0000","last_change":"2024-09-10T18:34:12.424471+0000","last_active":"2024-09-10T18:36:09.297111+0000","last_peered":"2024-09-10T18:36:09.297111+0000","last_clean":"2024-09-10T18:36:09.297111+0000","last_became_active":"2024-09-10T18:34:12.424068+0000","last_became_peered":"2024-09-10T18:34:12.424068+0000","last_unstale":"2024-09-10T18:36:09.297111+0000","last_undegraded":"2024-09-10T18:36:09.297111+0000","last_fullsized":"2024-09-10T18:36:09.297111+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:52.926536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298525+0000","last_change":"2024-09-10T18:34:10.397588+0000","last_active":"2024-09-10T18:36:09.298525+0000","last_peered":"2024-09-10T18:36:09.298525+0000","last_clean":"2024-09-10T18:36:09.298525+0000","last_became_active":"2024-09-10T18:34:10.397303+0000","last_became_peered":"2024-09-10T18:34:10.397303+0000","last_unstale":"2024-09-10T18:36:09.298525+0000","last_undegraded":"2024-09-10T18:36:09.298525+0000","last_fullsized":"2024-09-10T18:36:09.298525+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:45.215055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'11","reported_seq":66,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.840435+0000","last_change":"2024-09-10T18:34:14.753975+0000","last_active":"2024-09-10T18:37:08.840435+0000","last_peered":"2024-09-10T18:37:08.840435+0000","last_clean":"2024-09-10T18:37:08.840435+0000","last_became_active":"2024-09-10T18:34:14.751285+0000","last_became_peered":"2024-09-10T18:34:14.751285+0000","last_unstale":"2024-09-10T18:37:08.840435+0000","last_undegraded":"2024-09-10T18:37:08.840435+0000","last_fullsized":"2024-09-10T18:37:08.840435+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:59.245646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285310+0000","last_change":"2024-09-10T18:34:16.772297+0000","last_active":"2024-09-10T18:36:09.285310+0000","last_peered":"2024-09-10T18:36:09.285310+0000","last_clean":"2024-09-10T18:36:09.285310+0000","last_became_active":"2024-09-10T18:34:16.771934+0000","last_became_peered":"2024-09-10T18:34:16.771934+0000","last_unstale":"2024-09-10T18:36:09.285310+0000","last_undegraded":"2024-09-10T18:36:09.285310+0000","last_fullsized":"2024-09-10T18:36:09.285310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:37:02.348334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292895+0000","last_change":"2024-09-10T18:34:12.410280+0000","last_active":"2024-09-10T18:36:09.292895+0000","last_peered":"2024-09-10T18:36:09.292895+0000","last_clean":"2024-09-10T18:36:09.292895+0000","last_became_active":"2024-09-10T18:34:12.409884+0000","last_became_peered":"2024-09-10T18:34:12.409884+0000","last_unstale":"2024-09-10T18:36:09.292895+0000","last_undegraded":"2024-09-10T18:36:09.292895+0000","last_fullsized":"2024-09-10T18:36:09.292895+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:13.201512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.235933+0000","last_change":"2024-09-10T18:34:10.384141+0000","last_active":"2024-09-10T18:35:09.235933+0000","last_peered":"2024-09-10T18:35:09.235933+0000","last_clean":"2024-09-10T18:35:09.235933+0000","last_became_active":"2024-09-10T18:34:10.383308+0000","last_became_peered":"2024-09-10T18:34:10.383308+0000","last_unstale":"2024-09-10T18:35:09.235933+0000","last_undegraded":"2024-09-10T18:35:09.235933+0000","last_fullsized":"2024-09-10T18:35:09.235933+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:53.745126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.299069+0000","last_change":"2024-09-10T18:34:14.762234+0000","last_active":"2024-09-10T18:36:09.299069+0000","last_peered":"2024-09-10T18:36:09.299069+0000","last_clean":"2024-09-10T18:36:09.299069+0000","last_became_active":"2024-09-10T18:34:14.761997+0000","last_became_peered":"2024-09-10T18:34:14.761997+0000","last_unstale":"2024-09-10T18:36:09.299069+0000","last_undegraded":"2024-09-10T18:36:09.299069+0000","last_fullsized":"2024-09-10T18:36:09.299069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:05:18.581518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.295845+0000","last_change":"2024-09-10T18:34:16.788187+0000","last_active":"2024-09-10T18:36:09.295845+0000","last_peered":"2024-09-10T18:36:09.295845+0000","last_clean":"2024-09-10T18:36:09.295845+0000","last_became_active":"2024-09-10T18:34:16.787854+0000","last_became_peered":"2024-09-10T18:34:16.787854+0000","last_unstale":"2024-09-10T18:36:09.295845+0000","last_undegraded":"2024-09-10T18:36:09.295845+0000","last_fullsized":"2024-09-10T18:36:09.295845+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:13.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"60'10","reported_seq":36,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284952+0000","last_change":"2024-09-10T18:34:12.413814+0000","last_active":"2024-09-10T18:36:09.284952+0000","last_peered":"2024-09-10T18:36:09.284952+0000","last_clean":"2024-09-10T18:36:09.284952+0000","last_became_active":"2024-09-10T18:34:12.413512+0000","last_became_peered":"2024-09-10T18:34:12.413512+0000","last_unstale":"2024-09-10T18:36:09.284952+0000","last_undegraded":"2024-09-10T18:36:09.284952+0000","last_fullsized":"2024-09-10T18:36:09.284952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:58.191220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301440+0000","last_change":"2024-09-10T18:34:10.389420+0000","last_active":"2024-09-10T18:36:09.301440+0000","last_peered":"2024-09-10T18:36:09.301440+0000","last_clean":"2024-09-10T18:36:09.301440+0000","last_became_active":"2024-09-10T18:34:10.389098+0000","last_became_peered":"2024-09-10T18:34:10.389098+0000","last_unstale":"2024-09-10T18:36:09.301440+0000","last_undegraded":"2024-09-10T18:36:09.301440+0000","last_fullsized":"2024-09-10T18:36:09.301440+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:07.696670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285659+0000","last_change":"2024-09-10T18:34:14.758307+0000","last_active":"2024-09-10T18:36:09.285659+0000","last_peered":"2024-09-10T18:36:09.285659+0000","last_clean":"2024-09-10T18:36:09.285659+0000","last_became_active":"2024-09-10T18:34:14.757967+0000","last_became_peered":"2024-09-10T18:34:14.757967+0000","last_unstale":"2024-09-10T18:36:09.285659+0000","last_undegraded":"2024-09-10T18:36:09.285659+0000","last_fullsized":"2024-09-10T18:36:09.285659+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:41.872278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302566+0000","last_change":"2024-09-10T18:34:16.788440+0000","last_active":"2024-09-10T18:36:09.302566+0000","last_peered":"2024-09-10T18:36:09.302566+0000","last_clean":"2024-09-10T18:36:09.302566+0000","last_became_active":"2024-09-10T18:34:16.788171+0000","last_became_peered":"2024-09-10T18:34:16.788171+0000","last_unstale":"2024-09-10T18:36:09.302566+0000","last_undegraded":"2024-09-10T18:36:09.302566+0000","last_fullsized":"2024-09-10T18:36:09.302566+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:40.980577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'6","reported_seq":30,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.299313+0000","last_change":"2024-09-10T18:34:12.429193+0000","last_active":"2024-09-10T18:36:09.299313+0000","last_peered":"2024-09-10T18:36:09.299313+0000","last_clean":"2024-09-10T18:36:09.299313+0000","last_became_active":"2024-09-10T18:34:12.428682+0000","last_became_peered":"2024-09-10T18:34:12.428682+0000","last_unstale":"2024-09-10T18:36:09.299313+0000","last_undegraded":"2024-09-10T18:36:09.299313+0000","last_fullsized":"2024-09-10T18:36:09.299313+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:07.319728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294547+0000","last_change":"2024-09-10T18:34:10.387597+0000","last_active":"2024-09-10T18:36:09.294547+0000","last_peered":"2024-09-10T18:36:09.294547+0000","last_clean":"2024-09-10T18:36:09.294547+0000","last_became_active":"2024-09-10T18:34:10.387347+0000","last_became_peered":"2024-09-10T18:34:10.387347+0000","last_unstale":"2024-09-10T18:36:09.294547+0000","last_undegraded":"2024-09-10T18:36:09.294547+0000","last_fullsized":"2024-09-10T18:36:09.294547+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:13.275399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302254+0000","last_change":"2024-09-10T18:34:14.761053+0000","last_active":"2024-09-10T18:36:09.302254+0000","last_peered":"2024-09-10T18:36:09.302254+0000","last_clean":"2024-09-10T18:36:09.302254+0000","last_became_active":"2024-09-10T18:34:14.760739+0000","last_became_peered":"2024-09-10T18:34:14.760739+0000","last_unstale":"2024-09-10T18:36:09.302254+0000","last_undegraded":"2024-09-10T18:36:09.302254+0000","last_fullsized":"2024-09-10T18:36:09.302254+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:47.732711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292010+0000","last_change":"2024-09-10T18:34:16.778948+0000","last_active":"2024-09-10T18:36:09.292010+0000","last_peered":"2024-09-10T18:36:09.292010+0000","last_clean":"2024-09-10T18:36:09.292010+0000","last_became_active":"2024-09-10T18:34:16.778597+0000","last_became_peered":"2024-09-10T18:34:16.778597+0000","last_unstale":"2024-09-10T18:36:09.292010+0000","last_undegraded":"2024-09-10T18:36:09.292010+0000","last_fullsized":"2024-09-10T18:36:09.292010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:08.778306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.294415+0000","last_change":"2024-09-10T18:34:12.413340+0000","last_active":"2024-09-10T18:36:09.294415+0000","last_peered":"2024-09-10T18:36:09.294415+0000","last_clean":"2024-09-10T18:36:09.294415+0000","last_became_active":"2024-09-10T18:34:12.412768+0000","last_became_peered":"2024-09-10T18:34:12.412768+0000","last_unstale":"2024-09-10T18:36:09.294415+0000","last_undegraded":"2024-09-10T18:36:09.294415+0000","last_fullsized":"2024-09-10T18:36:09.294415+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:32.071499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.237334+0000","last_change":"2024-09-10T18:34:10.378473+0000","last_active":"2024-09-10T18:35:09.237334+0000","last_peered":"2024-09-10T18:35:09.237334+0000","last_clean":"2024-09-10T18:35:09.237334+0000","last_became_active":"2024-09-10T18:34:10.378166+0000","last_became_peered":"2024-09-10T18:34:10.378166+0000","last_unstale":"2024-09-10T18:35:09.237334+0000","last_undegraded":"2024-09-10T18:35:09.237334+0000","last_fullsized":"2024-09-10T18:35:09.237334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:29.996352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302715+0000","last_change":"2024-09-10T18:34:14.757400+0000","last_active":"2024-09-10T18:36:09.302715+0000","last_peered":"2024-09-10T18:36:09.302715+0000","last_clean":"2024-09-10T18:36:09.302715+0000","last_became_active":"2024-09-10T18:34:14.756767+0000","last_became_peered":"2024-09-10T18:34:14.756767+0000","last_unstale":"2024-09-10T18:36:09.302715+0000","last_undegraded":"2024-09-10T18:36:09.302715+0000","last_fullsized":"2024-09-10T18:36:09.302715+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:43.490199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304020+0000","last_change":"2024-09-10T18:34:17.073902+0000","last_active":"2024-09-10T18:36:09.304020+0000","last_peered":"2024-09-10T18:36:09.304020+0000","last_clean":"2024-09-10T18:36:09.304020+0000","last_became_active":"2024-09-10T18:34:17.073029+0000","last_became_peered":"2024-09-10T18:34:17.073029+0000","last_unstale":"2024-09-10T18:36:09.304020+0000","last_undegraded":"2024-09-10T18:36:09.304020+0000","last_fullsized":"2024-09-10T18:36:09.304020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:28.392682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.292616+0000","last_change":"2024-09-10T18:34:16.780401+0000","last_active":"2024-09-10T18:36:09.292616+0000","last_peered":"2024-09-10T18:36:09.292616+0000","last_clean":"2024-09-10T18:36:09.292616+0000","last_became_active":"2024-09-10T18:34:16.780072+0000","last_became_peered":"2024-09-10T18:34:16.780072+0000","last_unstale":"2024-09-10T18:36:09.292616+0000","last_undegraded":"2024-09-10T18:36:09.292616+0000","last_fullsized":"2024-09-10T18:36:09.292616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.732582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"60'15","reported_seq":46,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303583+0000","last_change":"2024-09-10T18:34:12.417300+0000","last_active":"2024-09-10T18:36:09.303583+0000","last_peered":"2024-09-10T18:36:09.303583+0000","last_clean":"2024-09-10T18:36:09.303583+0000","last_became_active":"2024-09-10T18:34:12.416444+0000","last_became_peered":"2024-09-10T18:34:12.416444+0000","last_unstale":"2024-09-10T18:36:09.303583+0000","last_undegraded":"2024-09-10T18:36:09.303583+0000","last_fullsized":"2024-09-10T18:36:09.303583+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:36.223910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"53'1","reported_seq":35,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.293771+0000","last_change":"2024-09-10T18:34:10.379655+0000","last_active":"2024-09-10T18:36:09.293771+0000","last_peered":"2024-09-10T18:36:09.293771+0000","last_clean":"2024-09-10T18:36:09.293771+0000","last_became_active":"2024-09-10T18:34:10.379349+0000","last_became_peered":"2024-09-10T18:34:10.379349+0000","last_unstale":"2024-09-10T18:36:09.293771+0000","last_undegraded":"2024-09-10T18:36:09.293771+0000","last_fullsized":"2024-09-10T18:36:09.293771+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:21:04.232115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.297144+0000","last_change":"2024-09-10T18:34:14.762873+0000","last_active":"2024-09-10T18:36:09.297144+0000","last_peered":"2024-09-10T18:36:09.297144+0000","last_clean":"2024-09-10T18:36:09.297144+0000","last_became_active":"2024-09-10T18:34:14.762609+0000","last_became_peered":"2024-09-10T18:34:14.762609+0000","last_unstale":"2024-09-10T18:36:09.297144+0000","last_undegraded":"2024-09-10T18:36:09.297144+0000","last_fullsized":"2024-09-10T18:36:09.297144+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:53.771040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.301221+0000","last_change":"2024-09-10T18:34:16.794863+0000","last_active":"2024-09-10T18:36:09.301221+0000","last_peered":"2024-09-10T18:36:09.301221+0000","last_clean":"2024-09-10T18:36:09.301221+0000","last_became_active":"2024-09-10T18:34:16.794619+0000","last_became_peered":"2024-09-10T18:34:16.794619+0000","last_unstale":"2024-09-10T18:36:09.301221+0000","last_undegraded":"2024-09-10T18:36:09.301221+0000","last_fullsized":"2024-09-10T18:36:09.301221+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:04.189834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.302387+0000","last_change":"2024-09-10T18:34:12.421836+0000","last_active":"2024-09-10T18:36:09.302387+0000","last_peered":"2024-09-10T18:36:09.302387+0000","last_clean":"2024-09-10T18:36:09.302387+0000","last_became_active":"2024-09-10T18:34:12.421554+0000","last_became_peered":"2024-09-10T18:34:12.421554+0000","last_unstale":"2024-09-10T18:36:09.302387+0000","last_undegraded":"2024-09-10T18:36:09.302387+0000","last_fullsized":"2024-09-10T18:36:09.302387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:41.279161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303668+0000","last_change":"2024-09-10T18:34:10.396213+0000","last_active":"2024-09-10T18:36:09.303668+0000","last_peered":"2024-09-10T18:36:09.303668+0000","last_clean":"2024-09-10T18:36:09.303668+0000","last_became_active":"2024-09-10T18:34:10.395688+0000","last_became_peered":"2024-09-10T18:34:10.395688+0000","last_unstale":"2024-09-10T18:36:09.303668+0000","last_undegraded":"2024-09-10T18:36:09.303668+0000","last_fullsized":"2024-09-10T18:36:09.303668+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:35.077708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":71,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:37:08.840614+0000","last_change":"2024-09-10T18:34:14.758358+0000","last_active":"2024-09-10T18:37:08.840614+0000","last_peered":"2024-09-10T18:37:08.840614+0000","last_clean":"2024-09-10T18:37:08.840614+0000","last_became_active":"2024-09-10T18:34:14.756787+0000","last_became_peered":"2024-09-10T18:34:14.756787+0000","last_unstale":"2024-09-10T18:37:08.840614+0000","last_undegraded":"2024-09-10T18:37:08.840614+0000","last_fullsized":"2024-09-10T18:37:08.840614+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:20.612765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.285015+0000","last_change":"2024-09-10T18:34:17.059458+0000","last_active":"2024-09-10T18:36:09.285015+0000","last_peered":"2024-09-10T18:36:09.285015+0000","last_clean":"2024-09-10T18:36:09.285015+0000","last_became_active":"2024-09-10T18:34:17.058421+0000","last_became_peered":"2024-09-10T18:34:17.058421+0000","last_unstale":"2024-09-10T18:36:09.285015+0000","last_undegraded":"2024-09-10T18:36:09.285015+0000","last_fullsized":"2024-09-10T18:36:09.285015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:07.162012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-10T18:35:09.236180+0000","last_change":"2024-09-10T18:34:10.379347+0000","last_active":"2024-09-10T18:35:09.236180+0000","last_peered":"2024-09-10T18:35:09.236180+0000","last_clean":"2024-09-10T18:35:09.236180+0000","last_became_active":"2024-09-10T18:34:10.379067+0000","last_became_peered":"2024-09-10T18:34:10.379067+0000","last_unstale":"2024-09-10T18:35:09.236180+0000","last_undegraded":"2024-09-10T18:35:09.236180+0000","last_fullsized":"2024-09-10T18:35:09.236180+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:21.812547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"60'5","reported_seq":31,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.298386+0000","last_change":"2024-09-10T18:34:12.424354+0000","last_active":"2024-09-10T18:36:09.298386+0000","last_peered":"2024-09-10T18:36:09.298386+0000","last_clean":"2024-09-10T18:36:09.298386+0000","last_became_active":"2024-09-10T18:34:12.424029+0000","last_became_peered":"2024-09-10T18:34:12.424029+0000","last_unstale":"2024-09-10T18:36:09.298386+0000","last_undegraded":"2024-09-10T18:36:09.298386+0000","last_fullsized":"2024-09-10T18:36:09.298386+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:23.902006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-10T18:36:10.663583+0000","last_change":"2024-09-10T18:36:10.663583+0000","last_active":"2024-09-10T18:36:10.663583+0000","last_peered":"2024-09-10T18:36:10.663583+0000","last_clean":"2024-09-10T18:36:10.663583+0000","last_became_active":"2024-09-10T18:36:10.662778+0000","last_became_peered":"2024-09-10T18:36:10.662778+0000","last_unstale":"2024-09-10T18:36:10.663583+0000","last_undegraded":"2024-09-10T18:36:10.663583+0000","last_fullsized":"2024-09-10T18:36:10.663583+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:22.082908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.304875+0000","last_change":"2024-09-10T18:34:17.073346+0000","last_active":"2024-09-10T18:36:09.304875+0000","last_peered":"2024-09-10T18:36:09.304875+0000","last_clean":"2024-09-10T18:36:09.304875+0000","last_became_active":"2024-09-10T18:34:17.072653+0000","last_became_peered":"2024-09-10T18:34:17.072653+0000","last_unstale":"2024-09-10T18:36:09.304875+0000","last_undegraded":"2024-09-10T18:36:09.304875+0000","last_fullsized":"2024-09-10T18:36:09.304875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:15.745253+0000","last_clean_scrub_stamp":"2024-09-10T18:34:15.745253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:33.753289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":23,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.305255+0000","last_change":"2024-09-10T18:34:10.388032+0000","last_active":"2024-09-10T18:36:09.305255+0000","last_peered":"2024-09-10T18:36:09.305255+0000","last_clean":"2024-09-10T18:36:09.305255+0000","last_became_active":"2024-09-10T18:34:10.387707+0000","last_became_peered":"2024-09-10T18:34:10.387707+0000","last_unstale":"2024-09-10T18:36:09.305255+0000","last_undegraded":"2024-09-10T18:36:09.305255+0000","last_fullsized":"2024-09-10T18:36:09.305255+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:09.358564+0000","last_clean_scrub_stamp":"2024-09-10T18:34:09.358564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:40.885222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"60'9","reported_seq":37,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.284506+0000","last_change":"2024-09-10T18:34:12.531543+0000","last_active":"2024-09-10T18:36:09.284506+0000","last_peered":"2024-09-10T18:36:09.284506+0000","last_clean":"2024-09-10T18:36:09.284506+0000","last_became_active":"2024-09-10T18:34:12.530541+0000","last_became_peered":"2024-09-10T18:34:12.530541+0000","last_unstale":"2024-09-10T18:36:09.284506+0000","last_undegraded":"2024-09-10T18:36:09.284506+0000","last_fullsized":"2024-09-10T18:36:09.284506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:11.385955+0000","last_clean_scrub_stamp":"2024-09-10T18:34:11.385955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:37.855220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T18:36:09.303593+0000","last_change":"2024-09-10T18:34:14.769494+0000","last_active":"2024-09-10T18:36:09.303593+0000","last_peered":"2024-09-10T18:36:09.303593+0000","last_clean":"2024-09-10T18:36:09.303593+0000","last_became_active":"2024-09-10T18:34:14.769231+0000","last_became_peered":"2024-09-10T18:34:14.769231+0000","last_unstale":"2024-09-10T18:36:09.303593+0000","last_undegraded":"2024-09-10T18:36:09.303593+0000","last_fullsized":"2024-09-10T18:36:09.303593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:34:13.729485+0000","last_clean_scrub_stamp":"2024-09-10T18:34:13.729485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:40.353540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":159,"num_read_kb":154,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298620,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446532,"statfs":{"total":95995035648,"available":95689248768,"internally_reserved":0,"allocated":1294336,"data_stored":855742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89100000000000001,"15min":0.88600000000000001},"min":{"1min":0.56799999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.29,"5min":1.431,"15min":1.431},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.879,"15min":0.89100000000000001},"min":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0489999999999999,"5min":1.214,"15min":1.214},"last":1.1279999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.96499999999999997,"15min":0.94299999999999995},"min":{"1min":0.77100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.4039999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.214,"5min":1.677,"15min":1.677},"last":0.86099999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.96399999999999997,"15min":0.94599999999999995},"min":{"1min":0.622,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3120000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.161},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.94799999999999995,"15min":0.95999999999999996},"min":{"1min":0.69799999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1899999999999999,"5min":1.323,"15min":1.323},"last":1.0349999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":0.96299999999999997,"15min":0.94699999999999995},"min":{"1min":0.69299999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2669999999999999,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.94099999999999995,"15min":0.94199999999999995},"min":{"1min":0.65500000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2350000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.94299999999999995}]},{"osd":4,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.878,"15min":0.876},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.161,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.94599999999999995,"15min":0.94999999999999996},"min":{"1min":0.61699999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.212,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.76700000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0680000000000001,"15min":1.0700000000000001},"min":{"1min":0.63300000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.4299999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":1},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.159,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.0029999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1279999999999999,"15min":1.1299999999999999},"min":{"1min":0.80300000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.4890000000000001,"5min":1.595,"15min":1.595},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0640000000000001,"15min":1.0740000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4339999999999999,"5min":1.643,"15min":1.643},"last":1.0960000000000001}]}]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298584,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446568,"statfs":{"total":95995035648,"available":95689285632,"internally_reserved":0,"allocated":1257472,"data_stored":853608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89000000000000001,"15min":0.85799999999999998},"min":{"1min":0.57699999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.294,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.236},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.875,"15min":0.877},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.73499999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.89400000000000002,"15min":0.91500000000000004},"min":{"1min":0.64800000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.24,"5min":1.488,"15min":1.488},"last":1.085},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.94899999999999995,"15min":0.93799999999999994},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.365,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.046}]},{"osd":2,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.95699999999999996,"15min":0.94299999999999995},"min":{"1min":0.70099999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.3340000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.008,"15min":1},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.1160000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":0.88900000000000001},"min":{"1min":0.61299999999999999,"5min":0.441,"15min":0.441},"max":{"1min":1.153,"5min":1.327,"15min":1.327},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1,"15min":0.98499999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.2490000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.87}]},{"osd":4,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86699999999999999,"15min":0.85399999999999998},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.216,"5min":1.266,"15min":1.266},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.92900000000000005},"min":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.97299999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.997,"15min":0.97499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3029999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.004,"15min":0.98899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.1870000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:36:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.135,"15min":1.1739999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.508,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.141,"15min":1.1579999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.425,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.208}]}]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298012,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447140,"statfs":{"total":95995035648,"available":95689871360,"internally_reserved":0,"allocated":671744,"data_stored":263962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86099999999999999,"15min":0.83099999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2549999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.872,"15min":0.88100000000000001},"min":{"1min":0.64000000000000001,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.272,"5min":1.379,"15min":1.379},"last":0.78500000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.89900000000000002,"15min":0.88200000000000001},"min":{"1min":0.64400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.202,"5min":1.226,"15min":1.226},"last":1.052},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.63300000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2150000000000001,"5min":1.278,"15min":1.278},"last":1.0089999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.89400000000000002,"15min":0.88300000000000001},"min":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1779999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.95399999999999996,"15min":0.93300000000000005},"min":{"1min":0.61899999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.323,"5min":1.359,"15min":1.359},"last":1.175}]},{"osd":3,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.93700000000000006,"15min":0.88900000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.25,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.95499999999999996,"15min":0.94699999999999995},"min":{"1min":0.69899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.27,"5min":1.542,"15min":1.542},"last":1.107}]},{"osd":4,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88900000000000001,"15min":0.83099999999999996},"min":{"1min":0.58699999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3049999999999999,"5min":1.423,"15min":1.423},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.97099999999999997,"15min":0.92300000000000004},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.306}]},{"osd":6,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0649999999999999,"15min":1.044},"min":{"1min":0.55900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5660000000000001,"5min":1.629,"15min":1.629},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0069999999999999,"15min":0.94799999999999995},"min":{"1min":0.71499999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.218}]},{"osd":7,"last update":"Tue Sep 10 18:36:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1279999999999999,"15min":1.161},"min":{"1min":0.70299999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.151},{"interface":"front","average":{"1min":1.105,"5min":1.1990000000000001,"15min":1.2330000000000001},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.508,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.6930000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986258,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298056,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447096,"statfs":{"total":95995035648,"available":95689826304,"internally_reserved":0,"allocated":712704,"data_stored":265771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.82299999999999995,"15min":0.81499999999999995},"min":{"1min":0.58899999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.137,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81399999999999995,"15min":0.83599999999999997},"min":{"1min":0.53300000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0609999999999999,"5min":1.163,"15min":1.163},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.86799999999999999,"15min":0.81899999999999995},"min":{"1min":0.75900000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1930000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.86199999999999999,"15min":0.83399999999999996},"min":{"1min":0.66800000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2310000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.85399999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.93200000000000005,"15min":0.89600000000000002},"min":{"1min":0.63600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1439999999999999,"5min":1.216,"15min":1.216},"last":1.111},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91200000000000003,"15min":0.89400000000000002},"min":{"1min":0.71099999999999997,"5min":0.64200000000000002,"15min":0.61599999999999999},"max":{"1min":1.29,"5min":1.405,"15min":1.405},"last":0.80400000000000005}]},{"osd":3,"last update":"Tue Sep 10 18:36:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.92500000000000004,"15min":0.89300000000000002},"min":{"1min":0.65100000000000002,"5min":0.62,"15min":0.62},"max":{"1min":1.1919999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.95599999999999996,"15min":0.93799999999999994},"min":{"1min":0.71299999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.0800000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.82699999999999996,"15min":0.752},"min":{"1min":0.64200000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0900000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.747},{"interface":"front","average":{"1min":1.048,"5min":0.97699999999999998,"15min":0.91200000000000003},"min":{"1min":0.72299999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3260000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.008}]},{"osd":6,"last update":"Tue Sep 10 18:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.88700000000000001,"15min":0.84499999999999997},"min":{"1min":0.61599999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.014,"15min":0.98199999999999998},"min":{"1min":0.86299999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2410000000000001,"5min":1.349,"15min":1.349},"last":1.052}]},{"osd":7,"last update":"Tue Sep 10 18:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.018,"15min":1.0389999999999999},"min":{"1min":0.59799999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.327,"5min":1.329,"15min":1.329},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.085,"15min":1.103},"min":{"1min":0.79600000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":1.135}]}]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298012,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447140,"statfs":{"total":95995035648,"available":95689871360,"internally_reserved":0,"allocated":671744,"data_stored":263935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39900000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93500000000000005,"5min":0.97299999999999998,"15min":1.819},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.748,"15min":0.72499999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.98899999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.93899999999999995}]},{"osd":1,"last update":"Tue Sep 10 18:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76100000000000001,"15min":0.73199999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0229999999999999,"5min":1.149,"15min":1.736},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1.0389999999999999,"15min":1.2130000000000001},"last":0.81299999999999994}]},{"osd":2,"last update":"Tue Sep 10 18:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.77800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.494},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.75800000000000001},"min":{"1min":0.39000000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.347},"last":0.77400000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.95499999999999996,"15min":0.92100000000000004},"min":{"1min":0.64100000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2370000000000001,"5min":4.2389999999999999,"15min":4.2389999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":0.91000000000000003},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.075}]},{"osd":5,"last update":"Tue Sep 10 18:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":0.94399999999999995},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.4339999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0329999999999999,"15min":1.0129999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":1.179}]},{"osd":6,"last update":"Tue Sep 10 18:36:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0620000000000001,"15min":1.046},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.321,"5min":1.464,"15min":1.464},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.044,"15min":1.0229999999999999},"min":{"1min":0.84299999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.365,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1559999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:36:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.1339999999999999,"15min":1.153},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.115,"15min":1.135},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.048}]}]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298012,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447140,"statfs":{"total":95995035648,"available":95689871360,"internally_reserved":0,"allocated":675840,"data_stored":264040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68600000000000005,"15min":0.61599999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85299999999999998,"5min":1.079,"15min":1.079},"last":0.44},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73599999999999999,"15min":0.64700000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.151,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.76000000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:37:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76800000000000002,"15min":0.68200000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1919999999999999,"5min":1.498,"15min":1.498},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.70399999999999996},"min":{"1min":0.54000000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Sep 10 18:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.82699999999999996,"15min":0.77200000000000002},"min":{"1min":0.58599999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.82799999999999996,"15min":0.77000000000000002},"min":{"1min":0.55000000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2729999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:36:51 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.92200000000000004,"15min":0.91000000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.095,"5min":1.0129999999999999,"15min":0.98199999999999998},"min":{"1min":0.72699999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.3879999999999999,"5min":1.5029999999999999,"15min":1.5069999999999999},"last":0.97799999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.96899999999999997,"15min":0.94699999999999995},"min":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.099,"15min":1.0509999999999999},"min":{"1min":0.88400000000000001,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:36:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.135,"15min":1.119},"min":{"1min":0.81399999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.085,"15min":1.0649999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.022}]},{"osd":7,"last update":"Tue Sep 10 18:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.095,"15min":1.0880000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.204,"15min":1.1910000000000001},"min":{"1min":0.94599999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.871}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298020,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447132,"statfs":{"total":95995035648,"available":95689863168,"internally_reserved":0,"allocated":679936,"data_stored":263737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:37:09 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65600000000000003,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.94399999999999995,"5min":1.056,"15min":1.056},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.77200000000000002,"15min":0.63200000000000001},"min":{"1min":0.5,"5min":0.45800000000000002,"15min":0.30399999999999999},"max":{"1min":1.2569999999999999,"5min":1.3100000000000001,"15min":1.5329999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:37:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.72099999999999997,"15min":0.65100000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.42599999999999999},"max":{"1min":1.0589999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79600000000000004,"15min":0.69999999999999996},"min":{"1min":0.59399999999999997,"5min":0.501,"15min":0.44},"max":{"1min":1.365,"5min":1.365,"15min":1.367},"last":0.83799999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84399999999999997,"15min":0.78600000000000003},"min":{"1min":0.625,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.4870000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.90000000000000002,"5min":1.028,"15min":1.028},"last":0.96199999999999997}]},{"osd":4,"last update":"Tue Sep 10 18:36:53 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.91800000000000004,"15min":0.86199999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.61799999999999999},"max":{"1min":1.3300000000000001,"5min":1.583,"15min":1.583},"last":1.194},{"interface":"front","average":{"1min":1.093,"5min":1.034,"15min":0.93799999999999994},"min":{"1min":0.73599999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.454,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.996}]},{"osd":5,"last update":"Tue Sep 10 18:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.94999999999999996,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.236,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.0840000000000001,"15min":1.05},"min":{"1min":0.82199999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.508,"5min":1.722,"15min":1.722},"last":1.044}]},{"osd":6,"last update":"Tue Sep 10 18:36:35 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.133,"15min":1.133},"min":{"1min":0.92500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.302,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.095},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3919999999999999,"5min":1.613,"15min":1.613},"last":1.411}]},{"osd":7,"last update":"Tue Sep 10 18:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.96299999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.383,"5min":1.484,"15min":1.484},"last":1.155},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0609999999999999,"15min":1.0549999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.3180000000000001,"5min":1.875,"15min":1.875},"last":1.3340000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298604,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446548,"statfs":{"total":95995035648,"available":95689265152,"internally_reserved":0,"allocated":1277952,"data_stored":854188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67100000000000004,"15min":0.59799999999999998},"min":{"1min":0.47399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.081},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.70999999999999996,"15min":0.63100000000000001},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.67900000000000005}]},{"osd":2,"last update":"Tue Sep 10 18:36:57 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72699999999999998,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.117,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81000000000000005,"15min":0.71399999999999997},"min":{"1min":0.57099999999999995,"5min":0.47499999999999998,"15min":0.41599999999999998},"max":{"1min":1.052,"5min":1.1619999999999999,"15min":1.1819999999999999},"last":0.97499999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:36:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79800000000000004,"15min":0.75900000000000001},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.878},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88800000000000001,"15min":0.80100000000000005},"min":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.45700000000000002},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.73799999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:36:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.90000000000000002,"15min":0.86099999999999999},"min":{"1min":0.65700000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.149},{"interface":"front","average":{"1min":1.044,"5min":1.0069999999999999,"15min":1.0089999999999999},"min":{"1min":0.70899999999999996,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.349,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:36:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.018,"15min":0.96399999999999997},"min":{"1min":0.76500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.252,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.0820000000000001,"15min":1.0620000000000001},"min":{"1min":0.82999999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3939999999999999,"5min":1.474,"15min":1.474},"last":1.3240000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:36:34 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.105,"15min":1.1379999999999999},"min":{"1min":0.69699999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.4219999999999999,"5min":1.548,"15min":1.548},"last":1.177},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.141,"15min":1.135},"min":{"1min":0.79000000000000004,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.258}]},{"osd":7,"last update":"Tue Sep 10 18:36:31 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.034,"15min":1.034},"min":{"1min":0.75,"5min":0.5,"15min":0.5},"max":{"1min":1.3700000000000001,"5min":1.458,"15min":1.458},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1359999999999999,"15min":1.1259999999999999},"min":{"1min":0.81699999999999995,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.425,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.232}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:37:20.283 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:37:20.284 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T18:37:20.284 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T18:37:20.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T18:37:20.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:20 smithi100 bash[18597]: cluster 2024-09-10T18:37:19.284125+0000 mgr.y (mgr.24512) 113 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:20.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:20 smithi100 bash[18597]: audit 2024-09-10T18:37:19.574182+0000 mgr.y (mgr.24512) 114 : audit [DBG] from='client.24686 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:37:20.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:20 smithi100 bash[31607]: cluster 2024-09-10T18:37:19.284125+0000 mgr.y (mgr.24512) 113 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:20.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:20 smithi100 bash[31607]: audit 2024-09-10T18:37:19.574182+0000 mgr.y (mgr.24512) 114 : audit [DBG] from='client.24686 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:37:20.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:20 smithi151 bash[22565]: cluster 2024-09-10T18:37:19.284125+0000 mgr.y (mgr.24512) 113 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:20.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:20 smithi151 bash[22565]: audit 2024-09-10T18:37:19.574182+0000 mgr.y (mgr.24512) 114 : audit [DBG] from='client.24686 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:37:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:37:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:21] "GET /metrics HTTP/1.1" 200 219675 "" "Prometheus/2.43.0" 2024-09-10T18:37:22.351 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:37:21 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:37:22.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:22 smithi151 bash[22565]: cluster 2024-09-10T18:37:21.285631+0000 mgr.y (mgr.24512) 115 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:22 smithi100 bash[18597]: cluster 2024-09-10T18:37:21.285631+0000 mgr.y (mgr.24512) 115 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:22.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:22 smithi100 bash[31607]: cluster 2024-09-10T18:37:21.285631+0000 mgr.y (mgr.24512) 115 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:23.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:23 smithi151 bash[22565]: audit 2024-09-10T18:37:21.985320+0000 mgr.y (mgr.24512) 116 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:23 smithi100 bash[18597]: audit 2024-09-10T18:37:21.985320+0000 mgr.y (mgr.24512) 116 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:23.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:23 smithi100 bash[31607]: audit 2024-09-10T18:37:21.985320+0000 mgr.y (mgr.24512) 116 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:24.676 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:37:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:37:24.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:24 smithi151 bash[22565]: cluster 2024-09-10T18:37:23.286315+0000 mgr.y (mgr.24512) 117 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:24.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:24 smithi151 bash[22565]: audit 2024-09-10T18:37:24.317983+0000 mon.c (mon.2) 82 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:24 smithi100 bash[18597]: cluster 2024-09-10T18:37:23.286315+0000 mgr.y (mgr.24512) 117 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:24 smithi100 bash[18597]: audit 2024-09-10T18:37:24.317983+0000 mon.c (mon.2) 82 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:24.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:24 smithi100 bash[31607]: cluster 2024-09-10T18:37:23.286315+0000 mgr.y (mgr.24512) 117 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:24.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:24 smithi100 bash[31607]: audit 2024-09-10T18:37:24.317983+0000 mon.c (mon.2) 82 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:25.063 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:37:26.659 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:26 smithi100 bash[18597]: cluster 2024-09-10T18:37:25.287660+0000 mgr.y (mgr.24512) 118 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:26.660 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:26 smithi100 bash[31607]: cluster 2024-09-10T18:37:25.287660+0000 mgr.y (mgr.24512) 118 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:26.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:26 smithi151 bash[22565]: cluster 2024-09-10T18:37:25.287660+0000 mgr.y (mgr.24512) 118 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- 172.21.15.100:0/1229644065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec00fc110 msgr2=0x7f6ec0108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1229644065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec00fc110 0x7f6ec0108530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc004980 tx=0x7f6ebc009da0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- 172.21.15.100:0/1229644065 shutdown_connections 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1229644065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec00fc110 0x7f6ec0108530 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1229644065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fb760 0x7f6ec00fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1229644065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec0101780 0x7f6ec0101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- 172.21.15.100:0/1229644065 >> 172.21.15.100:0/1229644065 conn(0x7f6ec0072400 msgr2=0x7f6ec00f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:28.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- 172.21.15.100:0/1229644065 shutdown_connections 2024-09-10T18:37:28.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- 172.21.15.100:0/1229644065 wait complete. 2024-09-10T18:37:28.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 Processor -- start 2024-09-10T18:37:28.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- start start 2024-09-10T18:37:28.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec00fb760 0x7f6ec0112ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:28.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fc110 0x7f6ec0113020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:28.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec0101780 0x7f6ec010cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:28.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec010d2c0 con 0x7f6ec0101780 2024-09-10T18:37:28.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec010d430 con 0x7f6ec00fb760 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec8621640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec010d5f0 con 0x7f6ec00fc110 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec6bbe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec0101780 0x7f6ec010cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fc110 0x7f6ec0113020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec63bd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec00fb760 0x7f6ec0112ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fc110 0x7f6ec0113020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34144/0 (socket says 172.21.15.100:34144) 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec63bd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec00fb760 0x7f6ec0112ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.100:60036/0 (socket says 172.21.15.100:60036) 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 -- 172.21.15.100:0/1353554922 learned_addr learned my addr 172.21.15.100:0/1353554922 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 -- 172.21.15.100:0/1353554922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec00fb760 msgr2=0x7f6ec0112ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec00fb760 0x7f6ec0112ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 -- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec0101780 msgr2=0x7f6ec010cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec0101780 0x7f6ec010cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 -- 172.21.15.100:0/1353554922 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80098b0 con 0x7f6ec00fc110 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec6bbe640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec0101780 0x7f6ec010cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6ec5bbc640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fc110 0x7f6ec0113020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8001f00 tx=0x7f6eb8004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.462+0000 7f6eb2ffd640 1 -- 172.21.15.100:0/1353554922 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800fea0 con 0x7f6ec00fc110 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.466+0000 7f6eb2ffd640 1 -- 172.21.15.100:0/1353554922 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f6eb8007d00 con 0x7f6ec00fc110 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.466+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc005040 con 0x7f6ec00fc110 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.466+0000 7f6eb2ffd640 1 -- 172.21.15.100:0/1353554922 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8015470 con 0x7f6ec00fc110 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.466+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec01199b0 con 0x7f6ec00fc110 2024-09-10T18:37:28.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.466+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea4005040 con 0x7f6ec00fc110 2024-09-10T18:37:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.466+0000 7f6eb2ffd640 1 -- 172.21.15.100:0/1353554922 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f6eb8007890 con 0x7f6ec00fc110 2024-09-10T18:37:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.470+0000 7f6eb2ffd640 1 --2- 172.21.15.100:0/1353554922 >> v2:172.21.15.100:6800/2155730476 conn(0x7f6eac06eec0 0x7f6eac071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:37:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.470+0000 7f6ec63bd640 1 --2- 172.21.15.100:0/1353554922 >> v2:172.21.15.100:6800/2155730476 conn(0x7f6eac06eec0 0x7f6eac071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:37:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.470+0000 7f6eb2ffd640 1 -- 172.21.15.100:0/1353554922 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6284+0+0 (secure 0 0 0) 0x7f6eb808e820 con 0x7f6ec00fc110 2024-09-10T18:37:28.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.470+0000 7f6ec63bd640 1 --2- 172.21.15.100:0/1353554922 >> v2:172.21.15.100:6800/2155730476 conn(0x7f6eac06eec0 0x7f6eac071370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009af0 tx=0x7f6eb400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:37:28.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.474+0000 7f6eb2ffd640 1 -- 172.21.15.100:0/1353554922 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6eb804f2a0 con 0x7f6ec00fc110 2024-09-10T18:37:28.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:28 smithi151 bash[22565]: cluster 2024-09-10T18:37:27.288383+0000 mgr.y (mgr.24512) 119 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:28.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.682+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6ea40052f0 con 0x7f6ec00fc110 2024-09-10T18:37:28.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.686+0000 7f6eb2ffd640 1 -- 172.21.15.100:0/1353554922 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6eb805b6c0 con 0x7f6ec00fc110 2024-09-10T18:37:28.689 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-10T18:37:28.689 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T18:37:28.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 >> v2:172.21.15.100:6800/2155730476 conn(0x7f6eac06eec0 msgr2=0x7f6eac071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:28.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1353554922 >> v2:172.21.15.100:6800/2155730476 conn(0x7f6eac06eec0 0x7f6eac071370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009af0 tx=0x7f6eb400b040 comp rx=0 tx=0).stop 2024-09-10T18:37:28.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fc110 msgr2=0x7f6ec0113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:28.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fc110 0x7f6ec0113020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8001f00 tx=0x7f6eb8004c50 comp rx=0 tx=0).stop 2024-09-10T18:37:28.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 shutdown_connections 2024-09-10T18:37:28.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1353554922 >> v2:172.21.15.100:6800/2155730476 conn(0x7f6eac06eec0 0x7f6eac071370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ec0101780 0x7f6ec010cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ec00fc110 0x7f6ec0113020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 --2- 172.21.15.100:0/1353554922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ec00fb760 0x7f6ec0112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:37:28.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 >> 172.21.15.100:0/1353554922 conn(0x7f6ec0072400 msgr2=0x7f6ec0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:28.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 shutdown_connections 2024-09-10T18:37:28.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-10T18:37:28.690+0000 7f6ec8621640 1 -- 172.21.15.100:0/1353554922 wait complete. 2024-09-10T18:37:28.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:28 smithi100 bash[18597]: cluster 2024-09-10T18:37:27.288383+0000 mgr.y (mgr.24512) 119 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:28.723 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:28 smithi100 bash[31607]: cluster 2024-09-10T18:37:27.288383+0000 mgr.y (mgr.24512) 119 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:29.471 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T18:37:29.472 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T18:37:29.472 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T18:37:29.491 INFO:tasks.workunit:Pulling workunits from ref fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:37:29.491 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T18:37:29.491 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:37:29.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:37:29.499 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T18:37:29.499 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:37:29.504 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-10T18:37:29.505 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-10T18:37:29.511 INFO:tasks.workunit:timeout=3h 2024-09-10T18:37:29.512 INFO:tasks.workunit:cleanup=True 2024-09-10T18:37:29.512 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/adk3798/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:37:29.563 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T18:37:29.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:29 smithi151 bash[22565]: audit 2024-09-10T18:37:28.689802+0000 mon.c (mon.2) 83 : audit [DBG] from='client.? 172.21.15.100:0/1353554922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:37:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:29 smithi100 bash[18597]: audit 2024-09-10T18:37:28.689802+0000 mon.c (mon.2) 83 : audit [DBG] from='client.? 172.21.15.100:0/1353554922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:37:29.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:29 smithi100 bash[31607]: audit 2024-09-10T18:37:28.689802+0000 mon.c (mon.2) 83 : audit [DBG] from='client.? 172.21.15.100:0/1353554922' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:37:30.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:30 smithi151 bash[22565]: cluster 2024-09-10T18:37:29.289104+0000 mgr.y (mgr.24512) 120 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:30.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:30 smithi100 bash[18597]: cluster 2024-09-10T18:37:29.289104+0000 mgr.y (mgr.24512) 120 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:30.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:30 smithi100 bash[31607]: cluster 2024-09-10T18:37:29.289104+0000 mgr.y (mgr.24512) 120 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:31.792 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:37:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:31] "GET /metrics HTTP/1.1" 200 219675 "" "Prometheus/2.43.0" 2024-09-10T18:37:32.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:37:31 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:37:32.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:32 smithi151 bash[22565]: cluster 2024-09-10T18:37:31.290679+0000 mgr.y (mgr.24512) 121 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:32.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:32 smithi100 bash[18597]: cluster 2024-09-10T18:37:31.290679+0000 mgr.y (mgr.24512) 121 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:32.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:32 smithi100 bash[31607]: cluster 2024-09-10T18:37:31.290679+0000 mgr.y (mgr.24512) 121 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:33.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:33 smithi151 bash[22565]: audit 2024-09-10T18:37:31.995650+0000 mgr.y (mgr.24512) 122 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:33.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:33 smithi100 bash[18597]: audit 2024-09-10T18:37:31.995650+0000 mgr.y (mgr.24512) 122 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:33.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:33 smithi100 bash[31607]: audit 2024-09-10T18:37:31.995650+0000 mgr.y (mgr.24512) 122 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:34.676 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:37:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:37:34.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:34 smithi151 bash[22565]: cluster 2024-09-10T18:37:33.291354+0000 mgr.y (mgr.24512) 123 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:34.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:34 smithi151 bash[22565]: audit 2024-09-10T18:37:34.366271+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:34.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:34 smithi100 bash[18597]: cluster 2024-09-10T18:37:33.291354+0000 mgr.y (mgr.24512) 123 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:34.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:34 smithi100 bash[18597]: audit 2024-09-10T18:37:34.366271+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:34 smithi100 bash[31607]: cluster 2024-09-10T18:37:33.291354+0000 mgr.y (mgr.24512) 123 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:34.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:34 smithi100 bash[31607]: audit 2024-09-10T18:37:34.366271+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:35.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:35 smithi151 bash[22565]: audit 2024-09-10T18:37:34.806169+0000 mon.c (mon.2) 85 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:35.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:35 smithi151 bash[22565]: audit 2024-09-10T18:37:34.807464+0000 mon.c (mon.2) 86 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:35.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:35 smithi151 bash[22565]: audit 2024-09-10T18:37:34.817663+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:37:35.792 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:35 smithi100 bash[18597]: audit 2024-09-10T18:37:34.806169+0000 mon.c (mon.2) 85 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:35 smithi100 bash[18597]: audit 2024-09-10T18:37:34.807464+0000 mon.c (mon.2) 86 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:35 smithi100 bash[18597]: audit 2024-09-10T18:37:34.817663+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:37:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:35 smithi100 bash[31607]: audit 2024-09-10T18:37:34.806169+0000 mon.c (mon.2) 85 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:35 smithi100 bash[31607]: audit 2024-09-10T18:37:34.807464+0000 mon.c (mon.2) 86 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:35 smithi100 bash[31607]: audit 2024-09-10T18:37:34.817663+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:37:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:36 smithi151 bash[22565]: cluster 2024-09-10T18:37:35.292835+0000 mgr.y (mgr.24512) 124 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:37.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:36 smithi100 bash[18597]: cluster 2024-09-10T18:37:35.292835+0000 mgr.y (mgr.24512) 124 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:36 smithi100 bash[31607]: cluster 2024-09-10T18:37:35.292835+0000 mgr.y (mgr.24512) 124 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:39.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:38 smithi151 bash[22565]: cluster 2024-09-10T18:37:37.293292+0000 mgr.y (mgr.24512) 125 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:39.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:38 smithi100 bash[18597]: cluster 2024-09-10T18:37:37.293292+0000 mgr.y (mgr.24512) 125 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:38 smithi100 bash[31607]: cluster 2024-09-10T18:37:37.293292+0000 mgr.y (mgr.24512) 125 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:40.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:39 smithi151 bash[22565]: audit 2024-09-10T18:37:39.318113+0000 mon.c (mon.2) 87 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:40.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:39 smithi100 bash[18597]: audit 2024-09-10T18:37:39.318113+0000 mon.c (mon.2) 87 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:40.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:39 smithi100 bash[31607]: audit 2024-09-10T18:37:39.318113+0000 mon.c (mon.2) 87 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:41.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:40 smithi151 bash[22565]: cluster 2024-09-10T18:37:39.293834+0000 mgr.y (mgr.24512) 126 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:41.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:40 smithi100 bash[18597]: cluster 2024-09-10T18:37:39.293834+0000 mgr.y (mgr.24512) 126 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:41.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:40 smithi100 bash[31607]: cluster 2024-09-10T18:37:39.293834+0000 mgr.y (mgr.24512) 126 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:41.792 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:37:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:41] "GET /metrics HTTP/1.1" 200 219666 "" "Prometheus/2.43.0" 2024-09-10T18:37:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:37:41 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:37:43.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:42 smithi151 bash[22565]: cluster 2024-09-10T18:37:41.295252+0000 mgr.y (mgr.24512) 127 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:43.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:42 smithi100 bash[31607]: cluster 2024-09-10T18:37:41.295252+0000 mgr.y (mgr.24512) 127 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:43.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:42 smithi100 bash[18597]: cluster 2024-09-10T18:37:41.295252+0000 mgr.y (mgr.24512) 127 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:44.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:43 smithi151 bash[22565]: audit 2024-09-10T18:37:42.006877+0000 mgr.y (mgr.24512) 128 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:44.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:43 smithi100 bash[18597]: audit 2024-09-10T18:37:42.006877+0000 mgr.y (mgr.24512) 128 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:43 smithi100 bash[31607]: audit 2024-09-10T18:37:42.006877+0000 mgr.y (mgr.24512) 128 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:44.845 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:44 smithi151 bash[22565]: cluster 2024-09-10T18:37:43.295800+0000 mgr.y (mgr.24512) 129 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:44.846 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:37:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:37:45.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:44 smithi100 bash[31607]: cluster 2024-09-10T18:37:43.295800+0000 mgr.y (mgr.24512) 129 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:44 smithi100 bash[18597]: cluster 2024-09-10T18:37:43.295800+0000 mgr.y (mgr.24512) 129 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:47.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:46 smithi151 bash[22565]: cluster 2024-09-10T18:37:45.297158+0000 mgr.y (mgr.24512) 130 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:47.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:46 smithi100 bash[18597]: cluster 2024-09-10T18:37:45.297158+0000 mgr.y (mgr.24512) 130 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:47.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:46 smithi100 bash[31607]: cluster 2024-09-10T18:37:45.297158+0000 mgr.y (mgr.24512) 130 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:49.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:48 smithi151 bash[22565]: cluster 2024-09-10T18:37:47.297608+0000 mgr.y (mgr.24512) 131 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:49.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:48 smithi100 bash[18597]: cluster 2024-09-10T18:37:47.297608+0000 mgr.y (mgr.24512) 131 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:49.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:48 smithi100 bash[31607]: cluster 2024-09-10T18:37:47.297608+0000 mgr.y (mgr.24512) 131 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:51.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:50 smithi151 bash[22565]: cluster 2024-09-10T18:37:49.298043+0000 mgr.y (mgr.24512) 132 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:51.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:50 smithi100 bash[18597]: cluster 2024-09-10T18:37:49.298043+0000 mgr.y (mgr.24512) 132 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:51.792 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:37:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:51] "GET /metrics HTTP/1.1" 200 219666 "" "Prometheus/2.43.0" 2024-09-10T18:37:51.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:51 smithi100 bash[31607]: cluster 2024-09-10T18:37:49.298043+0000 mgr.y (mgr.24512) 132 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:37:52.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:37:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:37:53.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:52 smithi151 bash[22565]: cluster 2024-09-10T18:37:51.298832+0000 mgr.y (mgr.24512) 133 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:53.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:52 smithi100 bash[31607]: cluster 2024-09-10T18:37:51.298832+0000 mgr.y (mgr.24512) 133 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:52 smithi100 bash[18597]: cluster 2024-09-10T18:37:51.298832+0000 mgr.y (mgr.24512) 133 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:54.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:53 smithi151 bash[22565]: audit 2024-09-10T18:37:52.018117+0000 mgr.y (mgr.24512) 134 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:54.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:53 smithi100 bash[18597]: audit 2024-09-10T18:37:52.018117+0000 mgr.y (mgr.24512) 134 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:54.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:53 smithi100 bash[31607]: audit 2024-09-10T18:37:52.018117+0000 mgr.y (mgr.24512) 134 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:37:54.874 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:37:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:37:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:37:55.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:54 smithi151 bash[22565]: cluster 2024-09-10T18:37:53.299247+0000 mgr.y (mgr.24512) 135 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:37:55.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:54 smithi151 bash[22565]: audit 2024-09-10T18:37:54.317490+0000 mon.c (mon.2) 88 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:55.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:54 smithi100 bash[18597]: cluster 2024-09-10T18:37:53.299247+0000 mgr.y (mgr.24512) 135 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:37:55.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:54 smithi100 bash[18597]: audit 2024-09-10T18:37:54.317490+0000 mon.c (mon.2) 88 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:54 smithi100 bash[31607]: cluster 2024-09-10T18:37:53.299247+0000 mgr.y (mgr.24512) 135 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:37:55.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:54 smithi100 bash[31607]: audit 2024-09-10T18:37:54.317490+0000 mon.c (mon.2) 88 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:37:57.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:56 smithi151 bash[22565]: cluster 2024-09-10T18:37:55.300024+0000 mgr.y (mgr.24512) 136 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:57.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:56 smithi100 bash[18597]: cluster 2024-09-10T18:37:55.300024+0000 mgr.y (mgr.24512) 136 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:57.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:56 smithi100 bash[31607]: cluster 2024-09-10T18:37:55.300024+0000 mgr.y (mgr.24512) 136 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:37:59.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:37:58 smithi151 bash[22565]: cluster 2024-09-10T18:37:57.300422+0000 mgr.y (mgr.24512) 137 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:37:59.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:37:58 smithi100 bash[18597]: cluster 2024-09-10T18:37:57.300422+0000 mgr.y (mgr.24512) 137 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:37:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:37:58 smithi100 bash[31607]: cluster 2024-09-10T18:37:57.300422+0000 mgr.y (mgr.24512) 137 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:38:01.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:00 smithi151 bash[22565]: cluster 2024-09-10T18:37:59.300820+0000 mgr.y (mgr.24512) 138 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:38:01.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:00 smithi100 bash[18597]: cluster 2024-09-10T18:37:59.300820+0000 mgr.y (mgr.24512) 138 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:38:01.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:00 smithi100 bash[31607]: cluster 2024-09-10T18:37:59.300820+0000 mgr.y (mgr.24512) 138 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:38:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:38:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:01] "GET /metrics HTTP/1.1" 200 219673 "" "Prometheus/2.43.0" 2024-09-10T18:38:02.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:38:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:38:03.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:02 smithi151 bash[22565]: cluster 2024-09-10T18:38:01.302182+0000 mgr.y (mgr.24512) 139 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:03.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:02 smithi100 bash[18597]: cluster 2024-09-10T18:38:01.302182+0000 mgr.y (mgr.24512) 139 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:03.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:02 smithi100 bash[31607]: cluster 2024-09-10T18:38:01.302182+0000 mgr.y (mgr.24512) 139 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:04.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:03 smithi151 bash[22565]: audit 2024-09-10T18:38:02.028268+0000 mgr.y (mgr.24512) 140 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:04.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:03 smithi100 bash[18597]: audit 2024-09-10T18:38:02.028268+0000 mgr.y (mgr.24512) 140 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:04.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:03 smithi100 bash[31607]: audit 2024-09-10T18:38:02.028268+0000 mgr.y (mgr.24512) 140 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:04.898 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:38:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:38:05.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:04 smithi151 bash[22565]: cluster 2024-09-10T18:38:03.302822+0000 mgr.y (mgr.24512) 141 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:05.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:04 smithi100 bash[18597]: cluster 2024-09-10T18:38:03.302822+0000 mgr.y (mgr.24512) 141 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:05.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:04 smithi100 bash[31607]: cluster 2024-09-10T18:38:03.302822+0000 mgr.y (mgr.24512) 141 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:07.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:06 smithi151 bash[22565]: cluster 2024-09-10T18:38:05.304169+0000 mgr.y (mgr.24512) 142 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:07.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:06 smithi100 bash[31607]: cluster 2024-09-10T18:38:05.304169+0000 mgr.y (mgr.24512) 142 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:06 smithi100 bash[18597]: cluster 2024-09-10T18:38:05.304169+0000 mgr.y (mgr.24512) 142 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:09.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:08 smithi151 bash[22565]: cluster 2024-09-10T18:38:07.304804+0000 mgr.y (mgr.24512) 143 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:09.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:08 smithi100 bash[31607]: cluster 2024-09-10T18:38:07.304804+0000 mgr.y (mgr.24512) 143 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:08 smithi100 bash[18597]: cluster 2024-09-10T18:38:07.304804+0000 mgr.y (mgr.24512) 143 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:10.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:09 smithi151 bash[22565]: audit 2024-09-10T18:38:09.318868+0000 mon.c (mon.2) 89 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:10.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:09 smithi100 bash[18597]: audit 2024-09-10T18:38:09.318868+0000 mon.c (mon.2) 89 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:10.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:09 smithi100 bash[31607]: audit 2024-09-10T18:38:09.318868+0000 mon.c (mon.2) 89 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:11.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:10 smithi151 bash[22565]: cluster 2024-09-10T18:38:09.305532+0000 mgr.y (mgr.24512) 144 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:11.450 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:10 smithi100 bash[18597]: cluster 2024-09-10T18:38:09.305532+0000 mgr.y (mgr.24512) 144 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:11.450 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:10 smithi100 bash[31607]: cluster 2024-09-10T18:38:09.305532+0000 mgr.y (mgr.24512) 144 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:11.451 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:38:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:11] "GET /metrics HTTP/1.1" 200 219676 "" "Prometheus/2.43.0" 2024-09-10T18:38:12.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:38:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:38:13.023 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to 'fad5356a94e6044ce1062d748daa5a0f2ed53a7d'. 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-10T18:38:13.024 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-09-10T18:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-10T18:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-09-10T18:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-10T18:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T18:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-10T18:38:13.025 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at fad5356a94e qa/upgrade: change pacific/octopus upgrade tests to use published image 2024-09-10T18:38:13.028 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T18:38:13.093 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T18:38:13.097 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:38:13.097 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T18:38:13.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:12 smithi151 bash[22565]: cluster 2024-09-10T18:38:11.306961+0000 mgr.y (mgr.24512) 145 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:13.292 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:12 smithi100 bash[18597]: cluster 2024-09-10T18:38:11.306961+0000 mgr.y (mgr.24512) 145 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:13.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:12 smithi100 bash[31607]: cluster 2024-09-10T18:38:11.306961+0000 mgr.y (mgr.24512) 145 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:13.990 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T18:38:14.066 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T18:38:14.128 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:38:14.130 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:38:14.130 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T18:38:14.193 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:38:14.199 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-10T18:38:14.199 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T18:38:14.203 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-10T18:38:14.205 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-10T18:38:14.205 DEBUG:teuthology.orchestra.run.smithi100:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fad5356a94e6044ce1062d748daa5a0f2ed53a7d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-10T18:38:14.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ parallel=1 2024-09-10T18:38:14.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' '' = --serial ']' 2024-09-10T18:38:14.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ color= 2024-09-10T18:38:14.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -t 1 ']' 2024-09-10T18:38:14.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-10T18:38:14.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ declare -A pids 2024-09-10T18:38:14.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.264 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_aio 2024-09-10T18:38:14.264 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_aio' 2024-09-10T18:38:14.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_aio 2024-09-10T18:38:14.268 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_aio 2024-09-10T18:38:14.268 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72588 2024-09-10T18:38:14.268 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_aio on pid 72588' 2024-09-10T18:38:14.269 INFO:tasks.workunit.client.0.smithi100.stdout:test api_aio on pid 72588 2024-09-10T18:38:14.269 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72588 2024-09-10T18:38:14.269 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.269 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.270 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-10T18:38:14.270 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_aio_pp 2024-09-10T18:38:14.270 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_aio_pp' 2024-09-10T18:38:14.271 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_aio_pp 2024-09-10T18:38:14.271 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.271 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.272 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.272 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.272 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.272 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_aio 2024-09-10T18:38:14.272 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-10T18:38:14.272 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_aio: /' 2024-09-10T18:38:14.274 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_aio_pp 2024-09-10T18:38:14.274 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72596 2024-09-10T18:38:14.275 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_aio_pp on pid 72596' 2024-09-10T18:38:14.275 INFO:tasks.workunit.client.0.smithi100.stdout:test api_aio_pp on pid 72596 2024-09-10T18:38:14.275 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72596 2024-09-10T18:38:14.276 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.276 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.276 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-10T18:38:14.276 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_io 2024-09-10T18:38:14.276 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_io' 2024-09-10T18:38:14.276 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_io 2024-09-10T18:38:14.276 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.277 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.277 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.277 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.277 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.277 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_aio_pp 2024-09-10T18:38:14.278 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-10T18:38:14.278 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-10T18:38:14.280 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_io 2024-09-10T18:38:14.280 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72604 2024-09-10T18:38:14.280 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_io on pid 72604' 2024-09-10T18:38:14.280 INFO:tasks.workunit.client.0.smithi100.stdout:test api_io on pid 72604 2024-09-10T18:38:14.281 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72604 2024-09-10T18:38:14.281 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.281 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.281 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-10T18:38:14.281 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_io_pp 2024-09-10T18:38:14.281 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_io_pp' 2024-09-10T18:38:14.282 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_io_pp 2024-09-10T18:38:14.282 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.282 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.283 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.283 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.283 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.283 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_io: /' 2024-09-10T18:38:14.283 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_io.log 2024-09-10T18:38:14.284 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_io 2024-09-10T18:38:14.285 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_io_pp 2024-09-10T18:38:14.285 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72612 2024-09-10T18:38:14.285 INFO:tasks.workunit.client.0.smithi100.stdout:test api_io_pp on pid 72612 2024-09-10T18:38:14.286 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_io_pp on pid 72612' 2024-09-10T18:38:14.286 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72612 2024-09-10T18:38:14.286 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.286 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.286 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-10T18:38:14.286 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_asio 2024-09-10T18:38:14.286 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_asio' 2024-09-10T18:38:14.287 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_asio 2024-09-10T18:38:14.287 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_asio 2024-09-10T18:38:14.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_io_pp 2024-09-10T18:38:14.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72619 2024-09-10T18:38:14.290 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-10T18:38:14.290 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_asio on pid 72619' 2024-09-10T18:38:14.290 INFO:tasks.workunit.client.0.smithi100.stdout:test api_asio on pid 72619 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72619 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_list 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_list' 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_list 2024-09-10T18:38:14.291 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.292 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:13 smithi100 bash[31607]: audit 2024-09-10T18:38:12.039683+0000 mgr.y (mgr.24512) 146 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:14.293 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:13 smithi100 bash[18597]: audit 2024-09-10T18:38:12.039683+0000 mgr.y (mgr.24512) 146 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:14.294 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.294 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.294 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.294 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-10T18:38:14.294 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_asio 2024-09-10T18:38:14.294 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_asio: /' 2024-09-10T18:38:14.295 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_list 2024-09-10T18:38:14.295 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72628 2024-09-10T18:38:14.295 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_list on pid 72628' 2024-09-10T18:38:14.295 INFO:tasks.workunit.client.0.smithi100.stdout:test api_list on pid 72628 2024-09-10T18:38:14.296 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72628 2024-09-10T18:38:14.296 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.296 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.296 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-10T18:38:14.296 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_lock 2024-09-10T18:38:14.296 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_lock' 2024-09-10T18:38:14.297 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_lock 2024-09-10T18:38:14.297 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.297 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.297 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.298 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.298 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.300 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_list 2024-09-10T18:38:14.300 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_list: /' 2024-09-10T18:38:14.301 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_list.log 2024-09-10T18:38:14.302 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_lock 2024-09-10T18:38:14.302 INFO:tasks.workunit.client.0.smithi100.stdout:test api_lock on pid 72636 2024-09-10T18:38:14.302 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72636 2024-09-10T18:38:14.302 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_lock on pid 72636' 2024-09-10T18:38:14.302 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72636 2024-09-10T18:38:14.303 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.303 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.303 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-10T18:38:14.307 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_lock_pp 2024-09-10T18:38:14.307 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_lock_pp' 2024-09-10T18:38:14.308 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_lock_pp 2024-09-10T18:38:14.308 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.308 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.308 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.308 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.308 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.309 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_lock: /' 2024-09-10T18:38:14.310 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_lock 2024-09-10T18:38:14.310 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-10T18:38:14.311 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_lock_pp 2024-09-10T18:38:14.311 INFO:tasks.workunit.client.0.smithi100.stdout:test api_lock_pp on pid 72650 2024-09-10T18:38:14.311 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72650 2024-09-10T18:38:14.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_lock_pp on pid 72650' 2024-09-10T18:38:14.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72650 2024-09-10T18:38:14.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-10T18:38:14.312 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_misc 2024-09-10T18:38:14.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_misc' 2024-09-10T18:38:14.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-10T18:38:14.315 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-10T18:38:14.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.316 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_lock_pp 2024-09-10T18:38:14.316 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_misc 2024-09-10T18:38:14.319 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_misc 2024-09-10T18:38:14.319 INFO:tasks.workunit.client.0.smithi100.stdout:test api_misc on pid 72689 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72689 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_misc on pid 72689' 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72689 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_misc_pp 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_misc_pp' 2024-09-10T18:38:14.320 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-10T18:38:14.321 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.321 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.321 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.322 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_misc_pp 2024-09-10T18:38:14.322 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.322 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.322 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_misc: /' 2024-09-10T18:38:14.322 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_misc 2024-09-10T18:38:14.323 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-10T18:38:14.326 INFO:tasks.workunit.client.0.smithi100.stdout:test api_misc_pp on pid 72714 2024-09-10T18:38:14.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_misc_pp 2024-09-10T18:38:14.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72714 2024-09-10T18:38:14.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_misc_pp on pid 72714' 2024-09-10T18:38:14.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72714 2024-09-10T18:38:14.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_tier_pp 2024-09-10T18:38:14.329 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_tier_pp' 2024-09-10T18:38:14.330 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-10T18:38:14.330 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.331 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_tier_pp 2024-09-10T18:38:14.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_misc_pp 2024-09-10T18:38:14.342 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-10T18:38:14.342 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-10T18:38:14.343 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_tier_pp 2024-09-10T18:38:14.343 INFO:tasks.workunit.client.0.smithi100.stdout:test api_tier_pp on pid 72749 2024-09-10T18:38:14.343 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72749 2024-09-10T18:38:14.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_tier_pp on pid 72749' 2024-09-10T18:38:14.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72749 2024-09-10T18:38:14.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-10T18:38:14.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.344 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_pool 2024-09-10T18:38:14.345 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_pool' 2024-09-10T18:38:14.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_pool 2024-09-10T18:38:14.349 INFO:tasks.workunit.client.0.smithi100.stdout:test api_pool on pid 72762 2024-09-10T18:38:14.350 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_pool 2024-09-10T18:38:14.350 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72762 2024-09-10T18:38:14.350 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_pool on pid 72762' 2024-09-10T18:38:14.350 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72762 2024-09-10T18:38:14.350 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.350 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.350 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_snapshots 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_snapshots' 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_tier_pp 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-10T18:38:14.351 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-10T18:38:14.352 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_snapshots 2024-09-10T18:38:14.352 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_pool 2024-09-10T18:38:14.354 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-10T18:38:14.358 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_snapshots 2024-09-10T18:38:14.358 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72782 2024-09-10T18:38:14.358 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_snapshots on pid 72782' 2024-09-10T18:38:14.359 INFO:tasks.workunit.client.0.smithi100.stdout:test api_snapshots on pid 72782 2024-09-10T18:38:14.360 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72782 2024-09-10T18:38:14.360 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.360 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.360 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_snapshots_pp 2024-09-10T18:38:14.361 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_snapshots_pp' 2024-09-10T18:38:14.361 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-10T18:38:14.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_snapshots_pp 2024-09-10T18:38:14.361 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_pool: /' 2024-09-10T18:38:14.366 INFO:tasks.workunit.client.0.smithi100.stdout:test api_snapshots_pp on pid 72794 2024-09-10T18:38:14.367 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.367 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.367 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_snapshots_pp 2024-09-10T18:38:14.367 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.367 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.367 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72794 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_snapshots_pp on pid 72794' 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72794 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_snapshots 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-10T18:38:14.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-10T18:38:14.369 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.369 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.369 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.369 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.375 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-10T18:38:14.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_stat 2024-09-10T18:38:14.375 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_stat' 2024-09-10T18:38:14.375 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-10T18:38:14.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_stat 2024-09-10T18:38:14.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.378 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_snapshots_pp 2024-09-10T18:38:14.387 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_stat 2024-09-10T18:38:14.388 INFO:tasks.workunit.client.0.smithi100.stdout:test api_stat on pid 72824 2024-09-10T18:38:14.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72824 2024-09-10T18:38:14.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_stat on pid 72824' 2024-09-10T18:38:14.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72824 2024-09-10T18:38:14.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-10T18:38:14.391 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_stat_pp 2024-09-10T18:38:14.391 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_stat_pp' 2024-09-10T18:38:14.397 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.397 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.397 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.397 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.400 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_stat 2024-09-10T18:38:14.400 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-10T18:38:14.400 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_stat: /' 2024-09-10T18:38:14.404 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.404 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_stat_pp 2024-09-10T18:38:14.409 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_stat_pp 2024-09-10T18:38:14.410 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72883 2024-09-10T18:38:14.410 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_stat_pp on pid 72883' 2024-09-10T18:38:14.410 INFO:tasks.workunit.client.0.smithi100.stdout:test api_stat_pp on pid 72883 2024-09-10T18:38:14.410 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72883 2024-09-10T18:38:14.410 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-10T18:38:14.410 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.411 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.411 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_watch_notify 2024-09-10T18:38:14.411 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_watch_notify' 2024-09-10T18:38:14.413 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.413 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.413 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.413 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.413 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-10T18:38:14.413 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_stat_pp 2024-09-10T18:38:14.419 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-10T18:38:14.422 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_watch_notify 2024-09-10T18:38:14.422 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.425 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_watch_notify 2024-09-10T18:38:14.425 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72910 2024-09-10T18:38:14.425 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_watch_notify on pid 72910' 2024-09-10T18:38:14.426 INFO:tasks.workunit.client.0.smithi100.stdout:test api_watch_notify on pid 72910 2024-09-10T18:38:14.426 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72910 2024-09-10T18:38:14.426 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.426 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:13 smithi151 bash[22565]: audit 2024-09-10T18:38:12.039683+0000 mgr.y (mgr.24512) 146 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:14.428 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-10T18:38:14.428 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_watch_notify_pp 2024-09-10T18:38:14.428 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_watch_notify_pp' 2024-09-10T18:38:14.433 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.433 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.433 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.433 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.433 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-10T18:38:14.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_watch_notify_pp 2024-09-10T18:38:14.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.438 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_watch_notify_pp 2024-09-10T18:38:14.438 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_watch_notify 2024-09-10T18:38:14.440 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-10T18:38:14.440 INFO:tasks.workunit.client.0.smithi100.stdout:test api_watch_notify_pp on pid 72940 2024-09-10T18:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72940 2024-09-10T18:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_watch_notify_pp on pid 72940' 2024-09-10T18:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72940 2024-09-10T18:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-10T18:38:14.441 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_cmd 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_cmd' 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-10T18:38:14.442 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_cmd 2024-09-10T18:38:14.443 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.449 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_cmd 2024-09-10T18:38:14.449 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72959 2024-09-10T18:38:14.449 INFO:tasks.workunit.client.0.smithi100.stdout:test api_cmd on pid 72959 2024-09-10T18:38:14.450 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_cmd on pid 72959' 2024-09-10T18:38:14.450 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72959 2024-09-10T18:38:14.451 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.451 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.451 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-10T18:38:14.451 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_cmd_pp 2024-09-10T18:38:14.451 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_cmd_pp' 2024-09-10T18:38:14.451 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.451 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_cmd_pp 2024-09-10T18:38:14.452 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_cmd_pp 2024-09-10T18:38:14.452 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72964 2024-09-10T18:38:14.453 INFO:tasks.workunit.client.0.smithi100.stdout:test api_cmd_pp on pid 72964 2024-09-10T18:38:14.455 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_cmd_pp on pid 72964' 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72964 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_service 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_service' 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_cmd 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_cmd: /' 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_cmd_pp 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_service 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.457 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-10T18:38:14.458 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-10T18:38:14.458 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_service 2024-09-10T18:38:14.459 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72975 2024-09-10T18:38:14.459 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_service on pid 72975' 2024-09-10T18:38:14.459 INFO:tasks.workunit.client.0.smithi100.stdout:test api_service on pid 72975 2024-09-10T18:38:14.461 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72975 2024-09-10T18:38:14.461 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.461 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.461 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-10T18:38:14.461 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_service_pp 2024-09-10T18:38:14.461 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_service_pp' 2024-09-10T18:38:14.461 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_service_pp 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_service 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_service.log 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_service: /' 2024-09-10T18:38:14.462 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_service_pp 2024-09-10T18:38:14.463 INFO:tasks.workunit.client.0.smithi100.stdout:test api_service_pp on pid 72983 2024-09-10T18:38:14.463 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72983 2024-09-10T18:38:14.463 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_service_pp on pid 72983' 2024-09-10T18:38:14.463 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-10T18:38:14.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72983 2024-09-10T18:38:14.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.464 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_c_write_operations 2024-09-10T18:38:14.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_c_write_operations' 2024-09-10T18:38:14.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_c_write_operations 2024-09-10T18:38:14.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_service_pp 2024-09-10T18:38:14.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-10T18:38:14.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-10T18:38:14.467 INFO:tasks.workunit.client.0.smithi100.stdout:test api_c_write_operations on pid 72991 2024-09-10T18:38:14.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_c_write_operations 2024-09-10T18:38:14.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72991 2024-09-10T18:38:14.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_c_write_operations on pid 72991' 2024-09-10T18:38:14.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72991 2024-09-10T18:38:14.468 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.468 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.468 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-10T18:38:14.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_c_read_operations 2024-09-10T18:38:14.468 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_c_read_operations' 2024-09-10T18:38:14.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_c_read_operations 2024-09-10T18:38:14.469 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.473 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.476 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.476 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.476 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.477 INFO:tasks.workunit.client.0.smithi100.stdout:test api_c_read_operations on pid 72999 2024-09-10T18:38:14.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_c_read_operations 2024-09-10T18:38:14.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72999 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_c_read_operations on pid 72999' 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=72999 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_c_write_operations 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-10T18:38:14.478 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_cls_remote_reads 2024-09-10T18:38:14.479 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_cls_remote_reads' 2024-09-10T18:38:14.479 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-10T18:38:14.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_cls_remote_reads 2024-09-10T18:38:14.483 INFO:tasks.workunit.client.0.smithi100.stdout:test api_cls_remote_reads on pid 73008 2024-09-10T18:38:14.484 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_cls_remote_reads 2024-09-10T18:38:14.484 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73008 2024-09-10T18:38:14.484 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_cls_remote_reads on pid 73008' 2024-09-10T18:38:14.484 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=73008 2024-09-10T18:38:14.484 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.484 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.484 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.485 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.485 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.485 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.485 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-10T18:38:14.489 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s list_parallel 2024-09-10T18:38:14.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' list_parallel' 2024-09-10T18:38:14.490 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_c_read_operations 2024-09-10T18:38:14.490 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-09-10T18:38:14.490 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-10T18:38:14.495 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.497 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.497 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-09-10T18:38:14.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo list_parallel 2024-09-10T18:38:14.499 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-09-10T18:38:14.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.499 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-09-10T18:38:14.502 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=list_parallel 2024-09-10T18:38:14.503 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73044 2024-09-10T18:38:14.503 INFO:tasks.workunit.client.0.smithi100.stdout:test list_parallel on pid 73044 2024-09-10T18:38:14.503 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test list_parallel on pid 73044' 2024-09-10T18:38:14.503 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=73044 2024-09-10T18:38:14.504 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.504 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.506 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s open_pools_parallel 2024-09-10T18:38:14.506 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' open_pools_parallel' 2024-09-10T18:38:14.511 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-10T18:38:14.513 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.513 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.513 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.513 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.513 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-10T18:38:14.514 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ list_parallel: /' 2024-09-10T18:38:14.514 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.514 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_list_parallel 2024-09-10T18:38:14.514 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo open_pools_parallel 2024-09-10T18:38:14.516 INFO:tasks.workunit.client.0.smithi100.stdout:test open_pools_parallel on pid 73076 2024-09-10T18:38:14.516 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=open_pools_parallel 2024-09-10T18:38:14.516 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73076 2024-09-10T18:38:14.516 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test open_pools_parallel on pid 73076' 2024-09-10T18:38:14.517 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=73076 2024-09-10T18:38:14.517 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:38:14.517 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.520 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-10T18:38:14.524 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s delete_pools_parallel 2024-09-10T18:38:14.524 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' delete_pools_parallel' 2024-09-10T18:38:14.525 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.525 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.525 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.525 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.526 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-10T18:38:14.526 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_open_pools_parallel 2024-09-10T18:38:14.526 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-10T18:38:14.527 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-09-10T18:38:14.527 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo delete_pools_parallel 2024-09-10T18:38:14.531 INFO:tasks.workunit.client.0.smithi100.stdout:test delete_pools_parallel on pid 73115 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=delete_pools_parallel 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73115 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test delete_pools_parallel on pid 73115' 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=73115 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ ret=0 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72588 2024-09-10T18:38:14.532 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72588 2024-09-10T18:38:14.537 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-10T18:38:14.538 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -z '' ']' 2024-09-10T18:38:14.539 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.539 INFO:tasks.workunit.client.0.smithi100.stderr:+ case $- in 2024-09-10T18:38:14.539 INFO:tasks.workunit.client.0.smithi100.stderr:+ return 2024-09-10T18:38:14.539 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-10T18:38:14.539 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-10T18:38:14.539 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_delete_pools_parallel 2024-09-10T18:38:14.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:38:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:38:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:16 smithi100 bash[18597]: cluster 2024-09-10T18:38:13.307698+0000 mgr.y (mgr.24512) 147 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:16 smithi100 bash[18597]: audit 2024-09-10T18:38:14.529869+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:16.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:16 smithi100 bash[31607]: cluster 2024-09-10T18:38:13.307698+0000 mgr.y (mgr.24512) 147 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:16.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:16 smithi100 bash[31607]: audit 2024-09-10T18:38:14.529869+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:16.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:16 smithi151 bash[22565]: cluster 2024-09-10T18:38:13.307698+0000 mgr.y (mgr.24512) 147 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:16.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:16 smithi151 bash[22565]: audit 2024-09-10T18:38:14.529869+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[31607]: audit 2024-09-10T18:38:15.211744+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.100:0/4176463563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[31607]: audit 2024-09-10T18:38:15.222841+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.100:0/4049546802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[31607]: cluster 2024-09-10T18:38:16.460449+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:38:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[31607]: audit 2024-09-10T18:38:16.481861+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[31607]: audit 2024-09-10T18:38:16.482181+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[31607]: audit 2024-09-10T18:38:16.482326+0000 mon.a (mon.0) 842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[18597]: audit 2024-09-10T18:38:15.211744+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.100:0/4176463563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[18597]: audit 2024-09-10T18:38:15.222841+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.100:0/4049546802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[18597]: cluster 2024-09-10T18:38:16.460449+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:38:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[18597]: audit 2024-09-10T18:38:16.481861+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[18597]: audit 2024-09-10T18:38:16.482181+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:18 smithi100 bash[18597]: audit 2024-09-10T18:38:16.482326+0000 mon.a (mon.0) 842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:18 smithi151 bash[22565]: audit 2024-09-10T18:38:15.211744+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.100:0/4176463563' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:18 smithi151 bash[22565]: audit 2024-09-10T18:38:15.222841+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.100:0/4049546802' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:18 smithi151 bash[22565]: cluster 2024-09-10T18:38:16.460449+0000 mon.a (mon.0) 839 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:38:18.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:18 smithi151 bash[22565]: audit 2024-09-10T18:38:16.481861+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:18.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:18 smithi151 bash[22565]: audit 2024-09-10T18:38:16.482181+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:18.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:18 smithi151 bash[22565]: audit 2024-09-10T18:38:16.482326+0000 mon.a (mon.0) 842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:38:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:21] "GET /metrics HTTP/1.1" 200 219676 "" "Prometheus/2.43.0" 2024-09-10T18:38:22.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:38:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] Global test environment set-up. 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (2 ms) 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-09-10T18:38:23.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (1 ms) 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (1 ms) 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (1 ms) 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1 ms) 2024-09-10T18:38:23.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] 12 tests from AsioRados (11 ms total) 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] Global test environment tear-down 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (9262 ms total) 2024-09-10T18:38:23.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: cluster 2024-09-10T18:38:15.309366+0000 mgr.y (mgr.24512) 148 : cluster [DBG] pgmap v99: 196 pgs: 46 creating+peering, 18 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: audit 2024-09-10T18:38:16.479272+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.100:0/3132937108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: audit 2024-09-10T18:38:16.509536+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.100:0/3400724520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: audit 2024-09-10T18:38:16.512195+0000 mon.c (mon.2) 93 : audit [DBG] from='client.? 172.21.15.100:0/2816048995' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: cluster 2024-09-10T18:38:17.310253+0000 mgr.y (mgr.24512) 149 : cluster [DBG] pgmap v101: 292 pgs: 46 creating+peering, 114 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: audit 2024-09-10T18:38:18.118550+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: cluster 2024-09-10T18:38:18.522281+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: audit 2024-09-10T18:38:18.532890+0000 mon.a (mon.0) 844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: audit 2024-09-10T18:38:18.539499+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[18597]: audit 2024-09-10T18:38:18.540991+0000 mon.a (mon.0) 846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: cluster 2024-09-10T18:38:15.309366+0000 mgr.y (mgr.24512) 148 : cluster [DBG] pgmap v99: 196 pgs: 46 creating+peering, 18 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: audit 2024-09-10T18:38:16.479272+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.100:0/3132937108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: audit 2024-09-10T18:38:16.509536+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.100:0/3400724520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: audit 2024-09-10T18:38:16.512195+0000 mon.c (mon.2) 93 : audit [DBG] from='client.? 172.21.15.100:0/2816048995' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: cluster 2024-09-10T18:38:17.310253+0000 mgr.y (mgr.24512) 149 : cluster [DBG] pgmap v101: 292 pgs: 46 creating+peering, 114 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: audit 2024-09-10T18:38:18.118550+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: cluster 2024-09-10T18:38:18.522281+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: audit 2024-09-10T18:38:18.532890+0000 mon.a (mon.0) 844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: audit 2024-09-10T18:38:18.539499+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:23 smithi100 bash[31607]: audit 2024-09-10T18:38:18.540991+0000 mon.a (mon.0) 846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: cluster 2024-09-10T18:38:15.309366+0000 mgr.y (mgr.24512) 148 : cluster [DBG] pgmap v99: 196 pgs: 46 creating+peering, 18 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: audit 2024-09-10T18:38:16.479272+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.100:0/3132937108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: audit 2024-09-10T18:38:16.509536+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.100:0/3400724520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: audit 2024-09-10T18:38:16.512195+0000 mon.c (mon.2) 93 : audit [DBG] from='client.? 172.21.15.100:0/2816048995' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: cluster 2024-09-10T18:38:17.310253+0000 mgr.y (mgr.24512) 149 : cluster [DBG] pgmap v101: 292 pgs: 46 creating+peering, 114 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: audit 2024-09-10T18:38:18.118550+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: cluster 2024-09-10T18:38:18.522281+0000 mon.a (mon.0) 843 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: audit 2024-09-10T18:38:18.532890+0000 mon.a (mon.0) 844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: audit 2024-09-10T18:38:18.539499+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:23.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:23 smithi151 bash[22565]: audit 2024-09-10T18:38:18.540991+0000 mon.a (mon.0) 846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:24.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:18.543040+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.100:0/3848778032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:18.545540+0000 mon.a (mon.0) 847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:18.545625+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:18.552056+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.100:0/196957021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:18.556626+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.100:0/4086083475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:18.558556+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.100:0/2996058608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:18.561170+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.100:0/3705484431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: cluster 2024-09-10T18:38:19.311389+0000 mgr.y (mgr.24512) 150 : cluster [DBG] pgmap v103: 452 pgs: 46 creating+peering, 274 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:24.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: cluster 2024-09-10T18:38:21.314693+0000 mgr.y (mgr.24512) 151 : cluster [DBG] pgmap v104: 452 pgs: 223 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:22.051121+0000 mgr.y (mgr.24512) 152 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.041315+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: cluster 2024-09-10T18:38:23.439594+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: cluster 2024-09-10T18:38:23.440307+0000 mon.a (mon.0) 850 : cluster [WRN] Health check failed: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.455279+0000 mon.a (mon.0) 851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.462099+0000 mon.a (mon.0) 852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.462420+0000 mon.a (mon.0) 853 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.462596+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.462824+0000 mon.a (mon.0) 855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.472298+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:24.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.484376+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:24.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.484533+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:24.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:23.500050+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:24.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: cluster 2024-09-10T18:38:24.328602+0000 mon.a (mon.0) 859 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:38:24.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:24 smithi151 bash[22565]: audit 2024-09-10T18:38:24.329573+0000 mon.a (mon.0) 860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:18.543040+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.100:0/3848778032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:18.545540+0000 mon.a (mon.0) 847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:18.545625+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:18.552056+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.100:0/196957021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:18.556626+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.100:0/4086083475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:18.558556+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.100:0/2996058608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:18.561170+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.100:0/3705484431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: cluster 2024-09-10T18:38:19.311389+0000 mgr.y (mgr.24512) 150 : cluster [DBG] pgmap v103: 452 pgs: 46 creating+peering, 274 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: cluster 2024-09-10T18:38:21.314693+0000 mgr.y (mgr.24512) 151 : cluster [DBG] pgmap v104: 452 pgs: 223 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:22.051121+0000 mgr.y (mgr.24512) 152 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.041315+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: cluster 2024-09-10T18:38:23.439594+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: cluster 2024-09-10T18:38:23.440307+0000 mon.a (mon.0) 850 : cluster [WRN] Health check failed: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.455279+0000 mon.a (mon.0) 851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:18.543040+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.100:0/3848778032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:18.545540+0000 mon.a (mon.0) 847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-72609-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:18.545625+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-72593-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:18.552056+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.100:0/196957021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:18.556626+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.100:0/4086083475' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:18.558556+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.100:0/2996058608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:18.561170+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.100:0/3705484431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: cluster 2024-09-10T18:38:19.311389+0000 mgr.y (mgr.24512) 150 : cluster [DBG] pgmap v103: 452 pgs: 46 creating+peering, 274 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: cluster 2024-09-10T18:38:21.314693+0000 mgr.y (mgr.24512) 151 : cluster [DBG] pgmap v104: 452 pgs: 223 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:22.051121+0000 mgr.y (mgr.24512) 152 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.041315+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:25.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: cluster 2024-09-10T18:38:23.439594+0000 mon.a (mon.0) 849 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: cluster 2024-09-10T18:38:23.440307+0000 mon.a (mon.0) 850 : cluster [WRN] Health check failed: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.455279+0000 mon.a (mon.0) 851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.462099+0000 mon.a (mon.0) 852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.462420+0000 mon.a (mon.0) 853 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.462596+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.462824+0000 mon.a (mon.0) 855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.472298+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.484376+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.484533+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:23.500050+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: cluster 2024-09-10T18:38:24.328602+0000 mon.a (mon.0) 859 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:38:25.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[18597]: audit 2024-09-10T18:38:24.329573+0000 mon.a (mon.0) 860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.462099+0000 mon.a (mon.0) 852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.462420+0000 mon.a (mon.0) 853 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.462596+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.462824+0000 mon.a (mon.0) 855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.472298+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:25.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.484376+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-72601-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.484533+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-72617-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:25.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:23.500050+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:25.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: cluster 2024-09-10T18:38:24.328602+0000 mon.a (mon.0) 859 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:38:25.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:24 smithi100 bash[31607]: audit 2024-09-10T18:38:24.329573+0000 mon.a (mon.0) 860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: cluster 2024-09-10T18:38:23.315846+0000 mgr.y (mgr.24512) 153 : cluster [DBG] pgmap v105: 452 pgs: 223 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.767282+0000 mon.a (mon.0) 861 : audit [INF] from='client.? 172.21.15.100:0/3286298592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-72795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.767773+0000 mon.a (mon.0) 862 : audit [INF] from='client.? 172.21.15.100:0/4116302683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-72806-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.768110+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.100:0/1524306223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.768401+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.100:0/656379181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-73032-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.768706+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-72633-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.769097+0000 mon.a (mon.0) 866 : audit [INF] from='client.? 172.21.15.100:0/1194092319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-72764-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.769413+0000 mon.a (mon.0) 867 : audit [INF] from='client.? 172.21.15.100:0/3685309934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-72929-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.769720+0000 mon.a (mon.0) 868 : audit [INF] from='client.? 172.21.15.100:0/2648099591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-73010-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.779441+0000 mon.a (mon.0) 869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.779536+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.779646+0000 mon.a (mon.0) 871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.779735+0000 mon.a (mon.0) 872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.779790+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.779956+0000 mon.a (mon.0) 874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.780034+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: cluster 2024-09-10T18:38:24.780690+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.784379+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.786648+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.100:0/4660899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.787414+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.100:0/2011814196' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.794587+0000 mon.a (mon.0) 877 : audit [DBG] from='client.? 172.21.15.100:0/482508201' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.800303+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.802150+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.803359+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.810530+0000 mon.a (mon.0) 881 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.819802+0000 mon.c (mon.2) 104 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:26.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:24.829730+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:25.251572+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:25.253004+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:25.660887+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:25.661661+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[18597]: audit 2024-09-10T18:38:25.662756+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: cluster 2024-09-10T18:38:23.315846+0000 mgr.y (mgr.24512) 153 : cluster [DBG] pgmap v105: 452 pgs: 223 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.767282+0000 mon.a (mon.0) 861 : audit [INF] from='client.? 172.21.15.100:0/3286298592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-72795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.767773+0000 mon.a (mon.0) 862 : audit [INF] from='client.? 172.21.15.100:0/4116302683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-72806-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.768110+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.100:0/1524306223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.768401+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.100:0/656379181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-73032-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.768706+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-72633-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.769097+0000 mon.a (mon.0) 866 : audit [INF] from='client.? 172.21.15.100:0/1194092319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-72764-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.769413+0000 mon.a (mon.0) 867 : audit [INF] from='client.? 172.21.15.100:0/3685309934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-72929-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.769720+0000 mon.a (mon.0) 868 : audit [INF] from='client.? 172.21.15.100:0/2648099591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-73010-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.779441+0000 mon.a (mon.0) 869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.779536+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.779646+0000 mon.a (mon.0) 871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.779735+0000 mon.a (mon.0) 872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.779790+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.779956+0000 mon.a (mon.0) 874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.780034+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: cluster 2024-09-10T18:38:24.780690+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.784379+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:26.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.786648+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.100:0/4660899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.787414+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.100:0/2011814196' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.794587+0000 mon.a (mon.0) 877 : audit [DBG] from='client.? 172.21.15.100:0/482508201' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.800303+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.802150+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.803359+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.810530+0000 mon.a (mon.0) 881 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.819802+0000 mon.c (mon.2) 104 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:24.829730+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:25.251572+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:25.253004+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:25.660887+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:25.661661+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:26.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:26 smithi100 bash[31607]: audit 2024-09-10T18:38:25.662756+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: cluster 2024-09-10T18:38:23.315846+0000 mgr.y (mgr.24512) 153 : cluster [DBG] pgmap v105: 452 pgs: 223 creating+peering, 33 unknown, 196 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.767282+0000 mon.a (mon.0) 861 : audit [INF] from='client.? 172.21.15.100:0/3286298592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-72795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.767773+0000 mon.a (mon.0) 862 : audit [INF] from='client.? 172.21.15.100:0/4116302683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-72806-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.768110+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.100:0/1524306223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.768401+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.100:0/656379181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-73032-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.768706+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-72633-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.769097+0000 mon.a (mon.0) 866 : audit [INF] from='client.? 172.21.15.100:0/1194092319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-72764-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.769413+0000 mon.a (mon.0) 867 : audit [INF] from='client.? 172.21.15.100:0/3685309934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-72929-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.769720+0000 mon.a (mon.0) 868 : audit [INF] from='client.? 172.21.15.100:0/2648099591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-73010-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.779441+0000 mon.a (mon.0) 869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.779536+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-72654-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.779646+0000 mon.a (mon.0) 871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-72887-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.779735+0000 mon.a (mon.0) 872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-72641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.779790+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-72846-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.779956+0000 mon.a (mon.0) 874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.780034+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: cluster 2024-09-10T18:38:24.780690+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.784379+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.786648+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.100:0/4660899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.787414+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.100:0/2011814196' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.794587+0000 mon.a (mon.0) 877 : audit [DBG] from='client.? 172.21.15.100:0/482508201' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.800303+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.802150+0000 mon.a (mon.0) 879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.803359+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.810530+0000 mon.a (mon.0) 881 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.819802+0000 mon.c (mon.2) 104 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:24.829730+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:25.251572+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:25.253004+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:25.660887+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:25.661661+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:26 smithi151 bash[22565]: audit 2024-09-10T18:38:25.662756+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-10T18:38:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] Global test environment set-up. 2024-09-10T18:38:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-10T18:38:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-10T18:38:26.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjects (87 ms) 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (31 ms) 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo2,foo3 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo2 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo3 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo4,foo5 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo4 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo5 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo6,foo7 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo7 2024-09-10T18:38:26.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo6 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo4 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo5 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo7 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo6 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo1 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo1 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo2 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo3 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (85 ms) 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 1 0 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 10 0 2024-09-10T18:38:26.718 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 13 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 7 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 14 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 0 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 15 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 11 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 5 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 8 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 6 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 3 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 4 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 12 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 9 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2 0 2024-09-10T18:38:26.719 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (99 ms) 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: x cursor=MIN 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=1 cursor=23:02547ec2:::1:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=10 cursor=23:52ea6a34:::10:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=13 cursor=23:566253c9:::13:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=7 cursor=23:5c6b0b28:::7:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=14 cursor=23:62a1935d:::14:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=0 cursor=23:6cac518f:::0:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=15 cursor=23:863748b0:::15:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=11 cursor=23:89d3ae78:::11:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=5 cursor=23:b29083e3:::5:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=8 cursor=23:bd63b0f1:::8:head 2024-09-10T18:38:26.720 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=6 cursor=23:c4fdafeb:::6:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=3 cursor=23:cfc208b3:::3:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=4 cursor=23:d83876eb:::4:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=12 cursor=23:de5d7c5f:::12:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=9 cursor=23:e960b815:::9:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=2 cursor=23:f905c69b:::2:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=1 cursor=23:02547ec2:::1:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:02547ec2:::1:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:02547ec2:::1:head -> 1 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=10 cursor=23:52ea6a34:::10:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:52ea6a34:::10:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:52ea6a34:::10:head -> 10 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=13 cursor=23:566253c9:::13:head 2024-09-10T18:38:26.721 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:566253c9:::13:head 2024-09-10T18:38:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:566253c9:::13:head -> 13 2024-09-10T18:38:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=7 cursor=23:5c6b0b28:::7:head 2024-09-10T18:38:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:5c6b0b28:::7:head 2024-09-10T18:38:26.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:5c6b0b28:::7:head -> 7 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=14 cursor=23:62a1935d:::14:head 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:62a1935d:::14:head 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:62a1935d:::14:head -> 14 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=0 cursor=23:6cac518f:::0:head 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:6cac518f:::0:head 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:6cac518f:::0:head -> 0 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=15 cursor=23:863748b0:::15:head 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:863748b0:::15:head 2024-09-10T18:38:26.930 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:863748b0:::15:head -> 15 2024-09-10T18:38:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=11 cursor=23:89d3ae78:::11:head 2024-09-10T18:38:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:89d3ae78:::11:head 2024-09-10T18:38:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:89d3ae78:::11:head -> 11 2024-09-10T18:38:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=5 cursor=23:b29083e3:::5:head 2024-09-10T18:38:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:b29083e3:::5:head 2024-09-10T18:38:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:b29083e3:::5:head -> 5 2024-09-10T18:38:26.931 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=8 cursor=23:bd63b0f1:::8:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:bd63b0f1:::8:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:bd63b0f1:::8:head -> 8 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=6 cursor=23:c4fdafeb:::6:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:c4fdafeb:::6:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:c4fdafeb:::6:head -> 6 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=3 cursor=23:cfc208b3:::3:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:cfc208b3:::3:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:cfc208b3:::3:head -> 3 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=4 cursor=23:d83876eb:::4:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:d83876eb:::4:head 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:d83876eb:::4:head -> 4 2024-09-10T18:38:26.932 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=12 cursor=23:de5d7c5f:::12:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:de5d7c5f:::12:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:de5d7c5f:::12:head -> 12 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=9 cursor=23:e960b815:::9:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:e960b815:::9:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:e960b815:::9:head -> 9 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=2 cursor=23:f905c69b:::2:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:f905c69b:::2:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:f905c69b:::2:head -> 2 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:52ea6a34:::10:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:52ea6a34:::10:head expected=23:52ea6a34:::10:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:52ea6a34:::10:head -> 10 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=10 expected=10 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:d83876eb:::4:head 2024-09-10T18:38:26.933 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:d83876eb:::4:head expected=23:d83876eb:::4:head 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:d83876eb:::4:head -> 4 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=4 expected=4 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:5c6b0b28:::7:head 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:5c6b0b28:::7:head expected=23:5c6b0b28:::7:head 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:5c6b0b28:::7:head -> 7 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=7 expected=7 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:cfc208b3:::3:head 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:cfc208b3:::3:head expected=23:cfc208b3:::3:head 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:cfc208b3:::3:head -> 3 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=3 expected=3 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:f905c69b:::2:head 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:f905c69b:::2:head expected=23:f905c69b:::2:head 2024-09-10T18:38:26.934 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:f905c69b:::2:head -> 2 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=2 expected=2 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:e960b815:::9:head 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:e960b815:::9:head expected=23:e960b815:::9:head 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:e960b815:::9:head -> 9 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=9 expected=9 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:de5d7c5f:::12:head 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:de5d7c5f:::12:head expected=23:de5d7c5f:::12:head 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:de5d7c5f:::12:head -> 12 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=12 expected=12 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:02547ec2:::1:head 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:02547ec2:::1:head expected=23:02547ec2:::1:head 2024-09-10T18:38:26.935 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:02547ec2:::1:head -> 1 2024-09-10T18:38:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=1 e api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-09-10T18:38:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-09-10T18:38:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-09-10T18:38:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-09-10T18:38:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (13197 ms) 2024-09-10T18:38:27.722 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (13197 ms total) 2024-09-10T18:38:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: 2024-09-10T18:38:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-09-10T18:38:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (13197 ms total) 2024-09-10T18:38:27.723 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-09-10T18:38:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: cluster 2024-09-10T18:38:25.321952+0000 mgr.y (mgr.24512) 154 : cluster [DBG] pgmap v108: 868 pgs: 480 creating+peering, 388 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2024-09-10T18:38:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791167+0000 mon.a (mon.0) 885 : audit [INF] from='client.? 172.21.15.100:0/3286298592' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-72795-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791284+0000 mon.a (mon.0) 886 : audit [INF] from='client.? 172.21.15.100:0/4116302683' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-72806-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791364+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.100:0/1524306223' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791444+0000 mon.a (mon.0) 888 : audit [INF] from='client.? 172.21.15.100:0/656379181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-73032-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791528+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-72633-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791613+0000 mon.a (mon.0) 890 : audit [INF] from='client.? 172.21.15.100:0/1194092319' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-72764-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791689+0000 mon.a (mon.0) 891 : audit [INF] from='client.? 172.21.15.100:0/3685309934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-72929-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:25.791779+0000 mon.a (mon.0) 892 : audit [INF] from='client.? 172.21.15.100:0/2648099591' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-73010-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.281619+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.281853+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.282564+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:28.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: cluster 2024-09-10T18:38:26.282788+0000 mon.a (mon.0) 896 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:38:28.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: cluster 2024-09-10T18:38:25.321952+0000 mgr.y (mgr.24512) 154 : cluster [DBG] pgmap v108: 868 pgs: 480 creating+peering, 388 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791167+0000 mon.a (mon.0) 885 : audit [INF] from='client.? 172.21.15.100:0/3286298592' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-72795-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791284+0000 mon.a (mon.0) 886 : audit [INF] from='client.? 172.21.15.100:0/4116302683' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-72806-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791364+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.100:0/1524306223' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791444+0000 mon.a (mon.0) 888 : audit [INF] from='client.? 172.21.15.100:0/656379181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-73032-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791528+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-72633-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791613+0000 mon.a (mon.0) 890 : audit [INF] from='client.? 172.21.15.100:0/1194092319' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-72764-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791689+0000 mon.a (mon.0) 891 : audit [INF] from='client.? 172.21.15.100:0/3685309934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-72929-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:25.791779+0000 mon.a (mon.0) 892 : audit [INF] from='client.? 172.21.15.100:0/2648099591' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-73010-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.281619+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.281853+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.282564+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:28.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: cluster 2024-09-10T18:38:26.282788+0000 mon.a (mon.0) 896 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.286091+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.287435+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.100:0/1540228064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.301201+0000 mon.a (mon.0) 897 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.305662+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.316849+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.321783+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.334197+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.337437+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:28.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.338135+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.338670+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.345317+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.346678+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.354067+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[18597]: audit 2024-09-10T18:38:26.804667+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.286091+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.287435+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.100:0/1540228064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.301201+0000 mon.a (mon.0) 897 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:28.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.305662+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.316849+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.321783+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.334197+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.337437+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.338135+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.338670+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.345317+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.346678+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.354067+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:27 smithi100 bash[31607]: audit 2024-09-10T18:38:26.804667+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: cluster 2024-09-10T18:38:25.321952+0000 mgr.y (mgr.24512) 154 : cluster [DBG] pgmap v108: 868 pgs: 480 creating+peering, 388 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 17 op/s 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791167+0000 mon.a (mon.0) 885 : audit [INF] from='client.? 172.21.15.100:0/3286298592' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-72795-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791284+0000 mon.a (mon.0) 886 : audit [INF] from='client.? 172.21.15.100:0/4116302683' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-72806-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791364+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.100:0/1524306223' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791444+0000 mon.a (mon.0) 888 : audit [INF] from='client.? 172.21.15.100:0/656379181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-73032-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791528+0000 mon.a (mon.0) 889 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-72633-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791613+0000 mon.a (mon.0) 890 : audit [INF] from='client.? 172.21.15.100:0/1194092319' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-72764-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791689+0000 mon.a (mon.0) 891 : audit [INF] from='client.? 172.21.15.100:0/3685309934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-72929-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:25.791779+0000 mon.a (mon.0) 892 : audit [INF] from='client.? 172.21.15.100:0/2648099591' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-73010-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.281619+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.281853+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.282564+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: cluster 2024-09-10T18:38:26.282788+0000 mon.a (mon.0) 896 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.286091+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.287435+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.100:0/1540228064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.301201+0000 mon.a (mon.0) 897 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.305662+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.316849+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.321783+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.334197+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.337437+0000 mon.a (mon.0) 900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:28.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.338135+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:28.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.338670+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.345317+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.346678+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:28.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.354067+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:27 smithi151 bash[22565]: audit 2024-09-10T18:38:26.804667+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-09-10T18:38:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-10T18:38:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-10T18:38:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-10T18:38:28.206 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (2040 ms) 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (20 ms) 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (11667 ms) 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (13727 ms total) 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (13727 ms total) 2024-09-10T18:38:28.207 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (2040 ms) 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-10T18:38:28.273 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (20 ms) 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (11702 ms) 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791167+0000 mon.a [INF] from='client.? 172.21.15.100:0/3286298592' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-72795-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791284+0000 mon.a [INF] from='client.? 172.21.15.100:0/4116302683' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-72806-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791364+0000 mon.a [INF] from='client.? 172.21.15.100:0/1524306223' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791444+0000 mon.a [INF] from='client.? 172.21.15.100:0/656379181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-73032-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791528+0000 mon.a [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-72633-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791613+0000 mon.a [INF] from='client.? 172.21.15.100:0/1194092319' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-72764-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791689+0000 mon.a [INF] from='client.? 172.21.15.100:0/3685309934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-72929-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:25.791779+0000 mon.a [INF] from='client.? 172.21.15.100:0/2648099591' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-73010-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.281619+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72966-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.281853+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72970-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:28.274 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.282564+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-72609-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.286091+0000 mon.c [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_nu api_c_read_operations: Running main() from gmock_main.cc 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (18 ms) 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (19 ms) 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-10T18:38:28.666 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (8 ms) 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (21 ms) 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (6 ms) 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (7 ms) 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (4 ms) 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-10T18:38:28.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (10 ms) 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (143 ms total) 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-10T18:38:28.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (14162 ms total) 2024-09-10T18:38:28.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout:m":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.287435+0000 mon.c [INF] from='client.? 172.21.15.100:0/1540228064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.301201+0000 mon.a [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.305662+0000 mon.a [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.316849+0000 mon.a [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.321783+0000 mon.b [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.334197+0000 mon.c [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:29.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.337437+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.338135+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.338670+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.345317+0000 mon.c [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.346678+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.354067+0000 mon.c [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.804667+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:29.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.811362+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:26.811563+0000 mon.a [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile- list_parallel: process_1_[73109]: starting. 2024-09-10T18:38:29.309 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[73109]: creating pool ceph_test_rados_list_parallel.smithi100-73072 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[73109]: created object 0... 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[73109]: created object 25... 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[73109]: created object 49... 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[73109]: finishing. 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[73109]: shutting down. 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[73110]: starting. 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[73110]: listing objects. 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[73110]: listed object 0... 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[73110]: listed object 25... 2024-09-10T18:38:29.310 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[73110]: saw 50 objects 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[73110]: shutting down. 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[73850]: starting. 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[73850]: creating pool ceph_test_rados_list_parallel.smithi100-73072 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[73850]: created object 0... 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[73850]: created object 25... 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[73850]: created object 49... 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[73850]: finishing. 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[73850]: shutting down. 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[73851]: starting. 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[73851]: listing objects. 2024-09-10T18:38:29.311 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[73851]: listed object 0... 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[73851]: listed object 25... 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[73851]: saw 43 objects 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[73851]: shutting down. 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[73852]: starting. 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[73852]: removed 25 objects... 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[73852]: removed half of the objects 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[73852]: removed 50 objects... 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[73852]: removed 50 objects 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[73852]: shutting down. 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.312 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[73969]: starting. 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[73969]: creating pool ceph_test_rados_list_parallel.smithi100-73072 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[73969]: created object 0... 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[73969]: created object 25... 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[73969]: created object 49... 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[73969]: finishing. 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[73969]: shutting down. 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[73970]: starting. 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[73970]: listing objects. 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[73970]: listed object 0... 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[73970]: listed object 25... 2024-09-10T18:38:29.313 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[73970]: saw 50 objects 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[73970]: shutting down. 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[73971]: starting. 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[73971]: added 25 objects... 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[73971]: added half of the objects 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[73971]: added 50 objects... 2024-09-10T18:38:29.314 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[73971]: added 50 objects 2024-09-10T18:38:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[73971]: shutting down. 2024-09-10T18:38:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:29.315 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[74140]: starting. 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:26.811362+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:26.811563+0000 mon.a (mon.0) 906 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:27.266060+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: cluster 2024-09-10T18:38:27.704229+0000 mon.a (mon.0) 908 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:27.717847+0000 mon.a (mon.0) 909 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:27.718450+0000 mon.a (mon.0) 910 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:27.719625+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:27.725727+0000 mon.a (mon.0) 912 : audit [DBG] from='client.? 172.21.15.100:0/3948155454' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:28.194645+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:28.194791+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:29.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:28.197837+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:28.198116+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:29.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:28.273464+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:29.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: audit 2024-09-10T18:38:28.648043+0000 mon.a (mon.0) 915 : audit [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:29.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: cluster 2024-09-10T18:38:28.649112+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:38:29.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:29 smithi151 bash[22565]: cluster 2024-09-10T18:38:28.649793+0000 mon.a (mon.0) 917 : cluster [WRN] Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:29.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:26.811362+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:29.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:26.811563+0000 mon.a (mon.0) 906 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:29.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:27.266060+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:29.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: cluster 2024-09-10T18:38:27.704229+0000 mon.a (mon.0) 908 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:38:29.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:27.717847+0000 mon.a (mon.0) 909 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:27.718450+0000 mon.a (mon.0) 910 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:27.719625+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:27.725727+0000 mon.a (mon.0) 912 : audit [DBG] from='client.? 172.21.15.100:0/3948155454' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:38:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:28.194645+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:28.194791+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:28.197837+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:28.198116+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:29.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:28.273464+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:29.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:26.811362+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-72946-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:29.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:26.811563+0000 mon.a (mon.0) 906 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:29.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:27.266060+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:29.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: cluster 2024-09-10T18:38:27.704229+0000 mon.a (mon.0) 908 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:38:29.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:27.717847+0000 mon.a (mon.0) 909 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:29.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:27.718450+0000 mon.a (mon.0) 910 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:29.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:27.719625+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:27.725727+0000 mon.a (mon.0) 912 : audit [DBG] from='client.? 172.21.15.100:0/3948155454' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:38:29.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:28.194645+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:29.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:28.194791+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:29.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:28.197837+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:29.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:28.198116+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:29.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:28.273464+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:29.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: audit 2024-09-10T18:38:28.648043+0000 mon.a (mon.0) 915 : audit [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:29.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: cluster 2024-09-10T18:38:28.649112+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:38:29.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[18597]: cluster 2024-09-10T18:38:28.649793+0000 mon.a (mon.0) 917 : cluster [WRN] Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:29.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: audit 2024-09-10T18:38:28.648043+0000 mon.a (mon.0) 915 : audit [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:29.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: cluster 2024-09-10T18:38:28.649112+0000 mon.a (mon.0) 916 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:38:29.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:29 smithi100 bash[31607]: cluster 2024-09-10T18:38:28.649793+0000 mon.a (mon.0) 917 : cluster [WRN] Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:30.495 INFO:tasks.workunit.client.0.smithi100.stdout:LibRadosIoECPP_smithi100-72617-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:30.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:27.266060+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-72593-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:30.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:27.718450+0000 mon.a [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:30.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:27.719625+0000 mon.a [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.194645+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-72609-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:30.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.194791+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-72846-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:30.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.197837+0000 mon.c [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.198116+0000 mon.c [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.273464+0000 mon.c [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.648043+0000 mon.a [INF] from='client.? 172.21.15.100:0/400742145' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.649793+0000 mon.a [WRN] Health check update: 15 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:28.273629+0000 client.admin [INF] onexx 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.097179+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.097549+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.122007+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:30.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.609944+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:30.755 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.619794+0000 mon.a [INF] from='client.? 172.21.15.100:0/2775561011' entity=' list_parallel: process_9_[74140]: creating pool ceph_test_rados_list_parallel.smithi100-73072 2024-09-10T18:38:30.755 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[74140]: created object 0... 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[74140]: created object 25... 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[74140]: created object 49... 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[74140]: finishing. 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[74140]: shutting down. 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: starting. 2024-09-10T18:38:30.756 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: listing objects. 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: listed object 0... 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: listed object 25... 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: listed object 50... 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: listed object 75... 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: saw 97 objects 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[74141]: shutting down. 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[74142]: starting. 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[74142]: added 25 objects... 2024-09-10T18:38:30.757 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[74142]: added half of the objects 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[74142]: added 50 objects... 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[74142]: added 50 objects 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[74142]: shutting down. 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[74144]: starting. 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[74144]: removed 25 objects... 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[74144]: removed half of the objects 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[74144]: removed 50 objects... 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[74144]: removed 50 objects 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[74144]: shutting down. 2024-09-10T18:38:30.758 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[74143]: starting. 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[74143]: added 25 objects... 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[74143]: added half of the objects 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[74143]: added 50 objects... 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[74143]: added 50 objects 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[74143]: shutting down. 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.759 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[74272]: starting. 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[74272]: creating pool ceph_test_rados_list_parallel.smithi100-73072 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[74272]: created object 0... 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[74272]: created object 25... 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[74272]: created object 49... 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[74272]: finishing. 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[74272]: shutting down. 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.760 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:30.761 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[74274]: starting. 2024-09-10T18:38:30.761 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[74274]: added 25 objects... 2024-09-10T18:38:30.761 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[74274]: added half of the objects 2024-09-10T18:38:30.761 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[74274]: added 50 objects... 2024-09-10T18:38:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: cluster 2024-09-10T18:38:27.323101+0000 mgr.y (mgr.24512) 155 : cluster [DBG] pgmap v111: 684 pgs: 104 unknown, 352 creating+peering, 228 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:38:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: cluster 2024-09-10T18:38:28.273629+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-09-10T18:38:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.096175+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.097179+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.097549+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.122007+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.609944+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.619794+0000 mon.a (mon.0) 923 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.620044+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.627147+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.100:0/3845701788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.789256+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.790099+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.790850+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.792029+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:38:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.792638+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: cluster 2024-09-10T18:38:27.323101+0000 mgr.y (mgr.24512) 155 : cluster [DBG] pgmap v111: 684 pgs: 104 unknown, 352 creating+peering, 228 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: cluster 2024-09-10T18:38:28.273629+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.096175+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.097179+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.097549+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.122007+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.609944+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.619794+0000 mon.a (mon.0) 923 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.620044+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.627147+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.100:0/3845701788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.789256+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.790099+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.790850+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.792029+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.792638+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.793368+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.794034+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:38:30.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.794629+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.795295+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:29.795962+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:30.038247+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:30.038935+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: cluster 2024-09-10T18:38:30.053609+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[18597]: audit 2024-09-10T18:38:30.057509+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.793368+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.794034+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.794629+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.795295+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:38:30.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:29.795962+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:38:30.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:30.038247+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:30.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:30.038935+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: cluster 2024-09-10T18:38:30.053609+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:38:30.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:30 smithi100 bash[31607]: audit 2024-09-10T18:38:30.057509+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: cluster 2024-09-10T18:38:27.323101+0000 mgr.y (mgr.24512) 155 : cluster [DBG] pgmap v111: 684 pgs: 104 unknown, 352 creating+peering, 228 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:38:30.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: cluster 2024-09-10T18:38:28.273629+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-09-10T18:38:30.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.096175+0000 mon.a (mon.0) 918 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:30.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.097179+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.097549+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:30.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.122007+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.609944+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.619794+0000 mon.a (mon.0) 923 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.620044+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.627147+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.100:0/3845701788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.789256+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.790099+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.790850+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.792029+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.792638+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:38:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.793368+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.794034+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.794629+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.795295+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:29.795962+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:30.038247+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:30.038935+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: cluster 2024-09-10T18:38:30.053609+0000 mon.a (mon.0) 925 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:38:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:30 smithi151 bash[22565]: audit 2024-09-10T18:38:30.057509+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:31] "GET /metrics HTTP/1.1" 200 274129 "" "Prometheus/2.43.0" 2024-09-10T18:38:32.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:38:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: cluster 2024-09-10T18:38:29.323925+0000 mgr.y (mgr.24512) 156 : cluster [DBG] pgmap v113: 660 pgs: 176 unknown, 256 creating+peering, 228 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.789497+0000 mgr.y (mgr.24512) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.790250+0000 mgr.y (mgr.24512) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.791008+0000 mgr.y (mgr.24512) 159 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.792180+0000 mgr.y (mgr.24512) 160 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.792785+0000 mgr.y (mgr.24512) 161 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.793550+0000 mgr.y (mgr.24512) 162 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:38:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.794177+0000 mgr.y (mgr.24512) 163 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.794801+0000 mgr.y (mgr.24512) 164 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.795476+0000 mgr.y (mgr.24512) 165 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:29.796123+0000 mgr.y (mgr.24512) 166 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.490290+0000 mon.a (mon.0) 927 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.501189+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.502627+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: cluster 2024-09-10T18:38:30.502773+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.503625+0000 mon.a (mon.0) 929 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.943825+0000 mon.a (mon.0) 930 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.950258+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.950533+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.951148+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:38:32.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:31 smithi151 bash[22565]: audit 2024-09-10T18:38:30.957633+0000 mon.a (mon.0) 934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: cluster 2024-09-10T18:38:29.323925+0000 mgr.y (mgr.24512) 156 : cluster [DBG] pgmap v113: 660 pgs: 176 unknown, 256 creating+peering, 228 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T18:38:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.789497+0000 mgr.y (mgr.24512) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:38:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.790250+0000 mgr.y (mgr.24512) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:38:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.791008+0000 mgr.y (mgr.24512) 159 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:38:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.792180+0000 mgr.y (mgr.24512) 160 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:38:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.792785+0000 mgr.y (mgr.24512) 161 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:38:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.793550+0000 mgr.y (mgr.24512) 162 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.794177+0000 mgr.y (mgr.24512) 163 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.794801+0000 mgr.y (mgr.24512) 164 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.795476+0000 mgr.y (mgr.24512) 165 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:29.796123+0000 mgr.y (mgr.24512) 166 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.490290+0000 mon.a (mon.0) 927 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.501189+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.502627+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: cluster 2024-09-10T18:38:30.502773+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.503625+0000 mon.a (mon.0) 929 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.943825+0000 mon.a (mon.0) 930 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.950258+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.950533+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.951148+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[18597]: audit 2024-09-10T18:38:30.957633+0000 mon.a (mon.0) 934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: cluster 2024-09-10T18:38:29.323925+0000 mgr.y (mgr.24512) 156 : cluster [DBG] pgmap v113: 660 pgs: 176 unknown, 256 creating+peering, 228 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.789497+0000 mgr.y (mgr.24512) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.790250+0000 mgr.y (mgr.24512) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.791008+0000 mgr.y (mgr.24512) 159 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.792180+0000 mgr.y (mgr.24512) 160 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.792785+0000 mgr.y (mgr.24512) 161 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.793550+0000 mgr.y (mgr.24512) 162 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.794177+0000 mgr.y (mgr.24512) 163 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.794801+0000 mgr.y (mgr.24512) 164 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:38:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.795476+0000 mgr.y (mgr.24512) 165 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:29.796123+0000 mgr.y (mgr.24512) 166 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.490290+0000 mon.a (mon.0) 927 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.501189+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.502627+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: cluster 2024-09-10T18:38:30.502773+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.503625+0000 mon.a (mon.0) 929 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.943825+0000 mon.a (mon.0) 930 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.950258+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.950533+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.951148+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:38:32.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:31 smithi100 bash[31607]: audit 2024-09-10T18:38:30.957633+0000 mon.a (mon.0) 934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-72846-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:32.299 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[74274]: added 50 objects 2024-09-10T18:38:32.299 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[74274]: shutting down. 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: starting. 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: listing objects. 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: listed object 0... 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: listed object 25... 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: listed object 50... 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: listed object 75... 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: listed object 100... 2024-09-10T18:38:32.300 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: listed object 125... 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: saw 150 objects 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[74273]: shutting down. 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-09-10T18:38:32.301 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******* SUCCESS ********** 2024-09-10T18:38:32.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: Running main() from gmock_main.cc 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: seed 72617 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (74 ms) 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (6 ms) 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-10T18:38:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (5 ms) 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (4 ms) 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (3 ms) 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (4 ms) 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (4 ms) 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (4 ms) 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (5 ms) 2024-09-10T18:38:32.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (7 ms) 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (5 ms) 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (6 ms) 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (16 ms) 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (6 ms) 2024-09-10T18:38:32.419 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (7 ms) 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (7 ms) 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (174 ms total) 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1891 ms) 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-10T18:38:32.420 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (21 ms) 2024-09-10T18:38:32.421 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-10T18:38:32.421 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-09-10T18:38:32.421 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-10T18:38:32.421 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-09-10T18:38:32.421 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-10T18:38:32.421 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (4 ms) 2024-09-10T18:38:33.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: cluster 2024-09-10T18:38:30.689594+0000 osd.3 (osd.3) 3 : cluster [DBG] 17.9 deep-scrub starts 2024-09-10T18:38:33.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: cluster 2024-09-10T18:38:30.692164+0000 osd.3 (osd.3) 4 : cluster [DBG] 17.9 deep-scrub ok 2024-09-10T18:38:33.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:30.949172+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:30.957847+0000 mon.a (mon.0) 935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:30.966261+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: cluster 2024-09-10T18:38:31.075752+0000 osd.7 (osd.7) 3 : cluster [DBG] 17.0 deep-scrub starts 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: cluster 2024-09-10T18:38:31.134087+0000 osd.7 (osd.7) 4 : cluster [DBG] 17.0 deep-scrub ok 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: cluster 2024-09-10T18:38:31.325084+0000 mgr.y (mgr.24512) 167 : cluster [DBG] pgmap v116: 612 pgs: 112 creating+peering, 72 unknown, 428 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 33 MiB/s wr, 579 op/s 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: cluster 2024-09-10T18:38:31.840062+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.843199+0000 mon.a (mon.0) 937 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.844449+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.845672+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.846434+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.853302+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.858167+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:38:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.864589+0000 mon.a (mon.0) 942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: cluster 2024-09-10T18:38:30.689594+0000 osd.3 (osd.3) 3 : cluster [DBG] 17.9 deep-scrub starts 2024-09-10T18:38:33.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: cluster 2024-09-10T18:38:30.692164+0000 osd.3 (osd.3) 4 : cluster [DBG] 17.9 deep-scrub ok 2024-09-10T18:38:33.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:30.949172+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:30.957847+0000 mon.a (mon.0) 935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:30.966261+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: cluster 2024-09-10T18:38:31.075752+0000 osd.7 (osd.7) 3 : cluster [DBG] 17.0 deep-scrub starts 2024-09-10T18:38:33.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: cluster 2024-09-10T18:38:31.134087+0000 osd.7 (osd.7) 4 : cluster [DBG] 17.0 deep-scrub ok 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: cluster 2024-09-10T18:38:31.325084+0000 mgr.y (mgr.24512) 167 : cluster [DBG] pgmap v116: 612 pgs: 112 creating+peering, 72 unknown, 428 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 33 MiB/s wr, 579 op/s 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: cluster 2024-09-10T18:38:31.840062+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.843199+0000 mon.a (mon.0) 937 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.844449+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.845672+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.846434+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.853302+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.858167+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.864589+0000 mon.a (mon.0) 942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.865146+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.865424+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.865890+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:31.866176+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.295354+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.295496+0000 mon.a (mon.0) 946 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.295540+0000 mon.a (mon.0) 947 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.295579+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.295818+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: cluster 2024-09-10T18:38:32.295881+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.296259+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.298974+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.299232+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.299465+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.100:0/1262768814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.865146+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.865424+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.865890+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:31.866176+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.295354+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.295496+0000 mon.a (mon.0) 946 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.295540+0000 mon.a (mon.0) 947 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.295579+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.295818+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: cluster 2024-09-10T18:38:32.295881+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.296259+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.298974+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.299232+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.549 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.299465+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.100:0/1262768814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.299688+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.100:0/525229010' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.299909+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.300814+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.302418+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.302675+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.303616+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.303883+0000 mon.a (mon.0) 955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.308502+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.317533+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.318273+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[31607]: audit 2024-09-10T18:38:32.318704+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.299688+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.100:0/525229010' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.299909+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.300814+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.302418+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.302675+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.303616+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.303883+0000 mon.a (mon.0) 955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.308502+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.317533+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.318273+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:33.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:33 smithi100 bash[18597]: audit 2024-09-10T18:38:32.318704+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: cluster 2024-09-10T18:38:30.689594+0000 osd.3 (osd.3) 3 : cluster [DBG] 17.9 deep-scrub starts 2024-09-10T18:38:33.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: cluster 2024-09-10T18:38:30.692164+0000 osd.3 (osd.3) 4 : cluster [DBG] 17.9 deep-scrub ok 2024-09-10T18:38:33.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:30.949172+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:30.957847+0000 mon.a (mon.0) 935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:30.966261+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: cluster 2024-09-10T18:38:31.075752+0000 osd.7 (osd.7) 3 : cluster [DBG] 17.0 deep-scrub starts 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: cluster 2024-09-10T18:38:31.134087+0000 osd.7 (osd.7) 4 : cluster [DBG] 17.0 deep-scrub ok 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: cluster 2024-09-10T18:38:31.325084+0000 mgr.y (mgr.24512) 167 : cluster [DBG] pgmap v116: 612 pgs: 112 creating+peering, 72 unknown, 428 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 33 MiB/s wr, 579 op/s 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: cluster 2024-09-10T18:38:31.840062+0000 mon.a (mon.0) 936 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.843199+0000 mon.a (mon.0) 937 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.844449+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.845672+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.846434+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.853302+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.858167+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.864589+0000 mon.a (mon.0) 942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.865146+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.865424+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.865890+0000 mon.c (mon.2) 131 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:31.866176+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.295354+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.295496+0000 mon.a (mon.0) 946 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.295540+0000 mon.a (mon.0) 947 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.295579+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.295818+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: cluster 2024-09-10T18:38:32.295881+0000 mon.a (mon.0) 950 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.296259+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.298974+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.299232+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.299465+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.100:0/1262768814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.299688+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.100:0/525229010' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.299909+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.300814+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.302418+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.302675+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.303616+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.303883+0000 mon.a (mon.0) 955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.308502+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.317533+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.318273+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:33.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:33 smithi151 bash[22565]: audit 2024-09-10T18:38:32.318704+0000 mon.a (mon.0) 958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.865 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] Lclient.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-72617-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:33.865 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.620044+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.865 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.627147+0000 mon.c [INF] from='client.? 172.21.15.100:0/3845701788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.865 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.789256+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:38:33.865 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.790099+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:38:33.865 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.790850+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.792029+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.792638+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.793368+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.794034+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.794629+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.795295+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:29.795962+0000 mon.c [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:30.038247+0000 mon.b [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:30.038935+0000 mon.b [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:30.057509+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:30.949172+0000 mon.b [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.866 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:30.957847+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:30.966261+0000 mon.c [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.844449+0000 mon.a [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.845672+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.846434+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.853302+0000 mon.c [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.858167+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.864589+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.865146+0000 mon.c [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.865424+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.865890+0000 mon.c [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.867 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:31.866176+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.295354+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.295496+0000 mon.a [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-72641-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.295540+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.295579+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.295818+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.296259+0000 mon.b [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.298974+0000 mon.c [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.299232+0000 mon.c [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.299465+0000 mon.c [INF] from='client.? 172.21.15.100:0/1262768814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.868 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.299688+0000 mon.c [INF] from='client.? 172.21.15.100:0/525229010' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.299909+0000 mon.c [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.300814+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.302418+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.302675+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.303616+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.303883+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.308502+0000 mon.c [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:33.869 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.317533+0000 mon.a [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.318273+0 delete_pools_parallel: process_1_[73144]: starting. 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[73144]: creating pool ceph_test_rados_delete_pools_parallel.smithi100-73125 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[73144]: created object 0... 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[73144]: created object 25... 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[73144]: created object 49... 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[73144]: finishing. 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[73144]: shutting down. 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_2_[73150]: starting. 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_2_[73150]: deleting pool ceph_test_rados_delete_pools_parallel.smithi100-73125 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_2_[73150]: shutting down. 2024-09-10T18:38:34.614 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[74033]: starting. 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[74033]: creating pool ceph_test_rados_delete_pools_parallel.smithi100-73125 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[74033]: created object 0... 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[74033]: created object 25... 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[74033]: created object 49... 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[74033]: finishing. 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[74033]: shutting down. 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[74036]: starting. 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[74036]: listing objects. 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[74036]: listed object 0... 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[74036]: listed object 25... 2024-09-10T18:38:34.615 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[74036]: saw 50 objects 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[74036]: shutting down. 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_4_[74035]: starting. 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_4_[74035]: deleting pool ceph_test_rados_delete_pools_parallel.smithi100-73125 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_4_[74035]: shutting down. 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:38:34.616 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-10T18:38:34.620 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[73119]: starting. 2024-09-10T18:38:34.620 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[73119]: creating pool ceph_test_rados_open_pools_parallel.smithi100-73104 2024-09-10T18:38:34.620 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[73119]: created object 0... 2024-09-10T18:38:34.620 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[73119]: created object 25... 2024-09-10T18:38:34.620 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[73119]: created object 49... 2024-09-10T18:38:34.620 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[73119]: finishing. 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[73119]: shutting down. 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[73121]: starting. 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[73121]: rados_pool_create. 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[73121]: rados_ioctx_create. 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[73121]: shutting down. 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[74032]: starting. 2024-09-10T18:38:34.621 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[74032]: creating pool ceph_test_rados_open_pools_parallel.smithi100-73104 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[74032]: created object 0... 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[74032]: created object 25... 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[74032]: created object 49... 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[74032]: finishing. 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[74032]: shutting down. 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[74034]: starting. 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[74034]: rados_pool_create. 2024-09-10T18:38:34.622 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[74034]: rados_ioctx_create. 2024-09-10T18:38:34.623 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[74034]: shutting down. 2024-09-10T18:38:34.623 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-09-10T18:38:34.623 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-09-10T18:38:34.623 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-10T18:38:34.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:34 smithi151 bash[22565]: audit 2024-09-10T18:38:32.061814+0000 mgr.y (mgr.24512) 168 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:34.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:34 smithi151 bash[22565]: audit 2024-09-10T18:38:33.178997+0000 mon.a (mon.0) 959 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:34.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:38:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:38:35.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:34 smithi100 bash[18597]: audit 2024-09-10T18:38:32.061814+0000 mgr.y (mgr.24512) 168 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:34 smithi100 bash[18597]: audit 2024-09-10T18:38:33.178997+0000 mon.a (mon.0) 959 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:35.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:34 smithi100 bash[31607]: audit 2024-09-10T18:38:32.061814+0000 mgr.y (mgr.24512) 168 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:35.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:34 smithi100 bash[31607]: audit 2024-09-10T18:38:33.178997+0000 mon.a (mon.0) 959 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:35.685 INFO:tasks.workunit.client.0.smithi100.stdout:000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:35.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:32.318704+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:35.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.731939+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:35.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.732189+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:35.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.732382+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.732560+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.733196+0000 mon.c [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.750641+0000 mon.c [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.865705+0000 mon.c [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:33.866064+0000 client.admin [INF] threexx 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.168553+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.169543+0000 mon.c [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.591959+0000 mon.b [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.592621+0000 mon.b [INF] from='client.? 172.21.15.100:0/3455921206' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.606143+0000 mon.a [WRN] Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:35.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.607091+0000 mon.c [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:35.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:33.325870+0000 mgr.y (mgr.24512) 169 : cluster [DBG] pgmap v119: 612 pgs: 47 creating+peering, 192 unknown, 373 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 541 op/s 2024-09-10T18:38:35.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.731939+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:35.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.732189+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:35.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.732382+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.732560+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.733196+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.750059+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.750641+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:33.865705+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:33.866064+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:33.897214+0000 osd.4 (osd.4) 3 : cluster [DBG] 17.3 deep-scrub starts 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:33.900525+0000 osd.4 (osd.4) 4 : cluster [DBG] 17.3 deep-scrub ok 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.091966+0000 osd.6 (osd.6) 3 : cluster [DBG] 17.6 deep-scrub starts 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.115999+0000 osd.7 (osd.7) 5 : cluster [DBG] 17.8 deep-scrub starts 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.119839+0000 osd.7 (osd.7) 6 : cluster [DBG] 17.8 deep-scrub ok 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.141083+0000 osd.6 (osd.6) 4 : cluster [DBG] 17.6 deep-scrub ok 2024-09-10T18:38:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.168553+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.169543+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.591959+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.592621+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.100:0/3455921206' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.605511+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.606143+0000 mon.a (mon.0) 966 : cluster [WRN] Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.607091+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.608063+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.613423+0000 mon.a (mon.0) 967 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.623541+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.623647+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.623840+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.624611+0000 mon.a (mon.0) 971 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:38:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.626134+0000 mon.a (mon.0) 972 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.642319+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.643050+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.643364+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.643796+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.644250+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.644539+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.645917+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.646153+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:34.867289+0000 mon.c (mon.2) 145 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.888308+0000 osd.0 (osd.0) 3 : cluster [DBG] 17.1 deep-scrub starts 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: cluster 2024-09-10T18:38:34.893307+0000 osd.0 (osd.0) 4 : cluster [DBG] 17.1 deep-scrub ok 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:35.296280+0000 mon.c (mon.2) 146 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:35.297571+0000 mon.c (mon.2) 147 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:35.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:35.307806+0000 mon.a (mon.0) 981 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:35.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:35 smithi151 bash[22565]: audit 2024-09-10T18:38:35.615121+0000 mon.a (mon.0) 982 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:36.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:33.325870+0000 mgr.y (mgr.24512) 169 : cluster [DBG] pgmap v119: 612 pgs: 47 creating+peering, 192 unknown, 373 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 541 op/s 2024-09-10T18:38:36.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.731939+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.732189+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.732382+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.732560+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.733196+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.750059+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.750641+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:33.865705+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:33.866064+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:33.897214+0000 osd.4 (osd.4) 3 : cluster [DBG] 17.3 deep-scrub starts 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:33.900525+0000 osd.4 (osd.4) 4 : cluster [DBG] 17.3 deep-scrub ok 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.091966+0000 osd.6 (osd.6) 3 : cluster [DBG] 17.6 deep-scrub starts 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.115999+0000 osd.7 (osd.7) 5 : cluster [DBG] 17.8 deep-scrub starts 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.119839+0000 osd.7 (osd.7) 6 : cluster [DBG] 17.8 deep-scrub ok 2024-09-10T18:38:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.141083+0000 osd.6 (osd.6) 4 : cluster [DBG] 17.6 deep-scrub ok 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.168553+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.169543+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:33.325870+0000 mgr.y (mgr.24512) 169 : cluster [DBG] pgmap v119: 612 pgs: 47 creating+peering, 192 unknown, 373 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 541 op/s 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.731939+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.732189+0000 mon.a (mon.0) 961 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.732382+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-72593-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.732560+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.733196+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.100:0/2581597768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.750059+0000 mon.c (mon.2) 139 : audit [DBG] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:38:36.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.750641+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:33.865705+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:33.866064+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:33.897214+0000 osd.4 (osd.4) 3 : cluster [DBG] 17.3 deep-scrub starts 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:33.900525+0000 osd.4 (osd.4) 4 : cluster [DBG] 17.3 deep-scrub ok 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.091966+0000 osd.6 (osd.6) 3 : cluster [DBG] 17.6 deep-scrub starts 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.115999+0000 osd.7 (osd.7) 5 : cluster [DBG] 17.8 deep-scrub starts 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.119839+0000 osd.7 (osd.7) 6 : cluster [DBG] 17.8 deep-scrub ok 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.141083+0000 osd.6 (osd.6) 4 : cluster [DBG] 17.6 deep-scrub ok 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.168553+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.169543+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:36.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.591959+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.592621+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.100:0/3455921206' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.605511+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.606143+0000 mon.a (mon.0) 966 : cluster [WRN] Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.607091+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.608063+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.613423+0000 mon.a (mon.0) 967 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.623541+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.623647+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.623840+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.624611+0000 mon.a (mon.0) 971 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.626134+0000 mon.a (mon.0) 972 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.642319+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:36.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.643050+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.643364+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.591959+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.592621+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.100:0/3455921206' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.605511+0000 mon.a (mon.0) 965 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.606143+0000 mon.a (mon.0) 966 : cluster [WRN] Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.607091+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.608063+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.613423+0000 mon.a (mon.0) 967 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.623541+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.623647+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:36.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.623840+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.624611+0000 mon.a (mon.0) 971 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.626134+0000 mon.a (mon.0) 972 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.642319+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.643050+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.643364+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.643796+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.644250+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.644539+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.645917+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.049 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.646153+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:34.867289+0000 mon.c (mon.2) 145 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.888308+0000 osd.0 (osd.0) 3 : cluster [DBG] 17.1 deep-scrub starts 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: cluster 2024-09-10T18:38:34.893307+0000 osd.0 (osd.0) 4 : cluster [DBG] 17.1 deep-scrub ok 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:35.296280+0000 mon.c (mon.2) 146 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:35.297571+0000 mon.c (mon.2) 147 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:35.307806+0000 mon.a (mon.0) 981 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[31607]: audit 2024-09-10T18:38:35.615121+0000 mon.a (mon.0) 982 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.643796+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.644250+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.644539+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.645917+0000 mon.a (mon.0) 979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.646153+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:36.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:34.867289+0000 mon.c (mon.2) 145 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:36.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.888308+0000 osd.0 (osd.0) 3 : cluster [DBG] 17.1 deep-scrub starts 2024-09-10T18:38:36.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: cluster 2024-09-10T18:38:34.893307+0000 osd.0 (osd.0) 4 : cluster [DBG] 17.1 deep-scrub ok 2024-09-10T18:38:36.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:35.296280+0000 mon.c (mon.2) 146 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:36.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:35.297571+0000 mon.c (mon.2) 147 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:36.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:35.307806+0000 mon.a (mon.0) 981 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:36.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:35 smithi100 bash[18597]: audit 2024-09-10T18:38:35.615121+0000 mon.a (mon.0) 982 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:36.988 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.608063+0000 mon.c [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "tes api_io: Running main() from gmock_main.cc 2024-09-10T18:38:36.988 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] Global test environment set-up. 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (40 ms) 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-09-10T18:38:36.989 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (26 ms) 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (5 ms) 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.Checksum (2 ms) 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (8 ms) 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-10T18:38:36.990 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (8 ms) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.TruncTest (7 ms) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (5 ms) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (5 ms) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.RmXattr (17 ms) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.XattrIter (7 ms) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 14 tests from LibRadosIo (136 ms total) 2024-09-10T18:38:36.991 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (4193 ms) 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (8 ms) 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (8 ms) 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-10T18:38:36.992 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (9 ms) 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (5 ms) 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (5 ms) 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (6 ms) 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (13 ms) 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (7 ms) 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 10 tests from LibRadosIoEC (4260 ms total) 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: 2024-09-10T18:38:36.993 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] Global test environment tear-down 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [==========] 24 tests from 2 test suites ran. (22690 ms total) 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ PASSED ] 24 tests. 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: Running main() from gmock_main.cc 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] Global test environment set-up. 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.Stat (73 ms) 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-10T18:38:36.994 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.StatNS (92 ms) 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 4 tests from LibRadosStat (167 ms total) 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1424 ms) 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (8 ms) 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-09-10T18:38:36.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1434 ms total) 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] Global test environment tear-down 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (22578 ms total) 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ PASSED ] 8 tests. 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout:ibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (7 ms) 2024-09-10T18:38:36.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (8 ms) 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (7 ms) 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (4 ms) 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (5 ms) 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (13 ms) 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-10T18:38:36.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (10 ms) 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (5 ms) 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (8 ms) 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (2005 ms total) 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-10T18:38:36.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (22692 ms total) 2024-09-10T18:38:36.999 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: cluster 2024-09-10T18:38:35.302144+0000 mgr.y (mgr.24512) 170 : cluster [DBG] pgmap v121: 636 pgs: 88 creating+peering, 548 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.627689+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.633691+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.633934+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.634177+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.634346+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.634941+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.635176+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.635454+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.635594+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: cluster 2024-09-10T18:38:35.635890+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:38:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.639017+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.639720+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.639899+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.644208+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2"}]: dispatch 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: cluster 2024-09-10T18:38:35.302144+0000 mgr.y (mgr.24512) 170 : cluster [DBG] pgmap v121: 636 pgs: 88 creating+peering, 548 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.627689+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.633691+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.633934+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.634177+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.634346+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.634941+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.635176+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.635454+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.635594+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: cluster 2024-09-10T18:38:35.635890+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.639017+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.639720+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.639899+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.644208+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2"}]: dispatch 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.644939+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:37.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.667398+0000 mon.a (mon.0) 992 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.667785+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.668087+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.668430+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.669362+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.669724+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.699787+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.701256+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: cluster 2024-09-10T18:38:35.701596+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: audit 2024-09-10T18:38:35.712723+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:38:37.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[18597]: cluster 2024-09-10T18:38:36.532202+0000 mon.a (mon.0) 1000 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.644939+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.667398+0000 mon.a (mon.0) 992 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.667785+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.668087+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.668430+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.669362+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.669724+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.699787+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.701256+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: cluster 2024-09-10T18:38:35.701596+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: audit 2024-09-10T18:38:35.712723+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:38:37.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:36 smithi100 bash[31607]: cluster 2024-09-10T18:38:36.532202+0000 mon.a (mon.0) 1000 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-10T18:38:37.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: cluster 2024-09-10T18:38:35.302144+0000 mgr.y (mgr.24512) 170 : cluster [DBG] pgmap v121: 636 pgs: 88 creating+peering, 548 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-09-10T18:38:37.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.627689+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.100:0/3651522626' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:38:37.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.633691+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:37.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.633934+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]': finished 2024-09-10T18:38:37.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.634177+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.634346+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.634941+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.635176+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.635454+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.635594+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: cluster 2024-09-10T18:38:35.635890+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.639017+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.639720+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.100:0/1739966653' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.639899+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.100:0/3605150558' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.644208+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2"}]: dispatch 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.644939+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.667398+0000 mon.a (mon.0) 992 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.667785+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.668087+0000 mon.a (mon.0) 994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.668430+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.669362+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.669724+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.699787+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.701256+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.100:0/3408419057' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: cluster 2024-09-10T18:38:35.701596+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: audit 2024-09-10T18:38:35.712723+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:38:37.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:36 smithi151 bash[22565]: cluster 2024-09-10T18:38:36.532202+0000 mon.a (mon.0) 1000 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-10T18:38:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: cluster 2024-09-10T18:38:36.136160+0000 osd.6 (osd.6) 5 : cluster [DBG] 17.5 deep-scrub starts 2024-09-10T18:38:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: cluster 2024-09-10T18:38:36.139669+0000 osd.6 (osd.6) 6 : cluster [DBG] 17.5 deep-scrub ok 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: cluster 2024-09-10T18:38:36.962952+0000 mon.a (mon.0) 1001 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.970838+0000 mon.a (mon.0) 1002 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.981135+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.983180+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.983244+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.983309+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.983367+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.983419+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.983622+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: cluster 2024-09-10T18:38:36.984419+0000 mon.a (mon.0) 1009 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.986527+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:38:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.986851+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.100:0/1942448514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.988519+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.989747+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: cluster 2024-09-10T18:38:36.136160+0000 osd.6 (osd.6) 5 : cluster [DBG] 17.5 deep-scrub starts 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: cluster 2024-09-10T18:38:36.139669+0000 osd.6 (osd.6) 6 : cluster [DBG] 17.5 deep-scrub ok 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: cluster 2024-09-10T18:38:36.962952+0000 mon.a (mon.0) 1001 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.970838+0000 mon.a (mon.0) 1002 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.981135+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.983180+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.983244+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:38:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.983309+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.983367+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.983419+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.983622+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: cluster 2024-09-10T18:38:36.984419+0000 mon.a (mon.0) 1009 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.986527+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.986851+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.100:0/1942448514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.988519+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.989747+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:38:38.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.994074+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.100:0/3190758578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-72593-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.994525+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.994948+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:36.996794+0000 mon.a (mon.0) 1014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:37.032557+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:37.032946+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[18597]: audit 2024-09-10T18:38:37.972621+0000 mon.a (mon.0) 1016 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.994074+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.100:0/3190758578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-72593-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.994525+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.994948+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:38:38.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:36.996794+0000 mon.a (mon.0) 1014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:37.032557+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]: dispatch 2024-09-10T18:38:38.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:37.032946+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]: dispatch 2024-09-10T18:38:38.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:38 smithi100 bash[31607]: audit 2024-09-10T18:38:37.972621+0000 mon.a (mon.0) 1016 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:38.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: cluster 2024-09-10T18:38:36.136160+0000 osd.6 (osd.6) 5 : cluster [DBG] 17.5 deep-scrub starts 2024-09-10T18:38:38.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: cluster 2024-09-10T18:38:36.139669+0000 osd.6 (osd.6) 6 : cluster [DBG] 17.5 deep-scrub ok 2024-09-10T18:38:38.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: cluster 2024-09-10T18:38:36.962952+0000 mon.a (mon.0) 1001 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:38.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.970838+0000 mon.a (mon.0) 1002 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:38.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.981135+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.983180+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-72617-22"}]': finished 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.983244+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-72601-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.983309+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.983367+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-72609-16"}]': finished 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.983419+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-72846-6"}]': finished 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.983622+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: cluster 2024-09-10T18:38:36.984419+0000 mon.a (mon.0) 1009 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.986527+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.986851+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.100:0/1942448514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.988519+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.989747+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:38:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.994074+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? 172.21.15.100:0/3190758578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-72593-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.994525+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.994948+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:38:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:36.996794+0000 mon.a (mon.0) 1014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:37.032557+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]: dispatch 2024-09-10T18:38:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:37.032946+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]: dispatch 2024-09-10T18:38:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:37 smithi151 bash[22565]: audit 2024-09-10T18:38:37.972621+0000 mon.a (mon.0) 1016 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:38.995 INFO:tasks.workunit.client.0.smithi100.stdout:tprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:38.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.623541+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-72654-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:38.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.623647+0000 mon.a [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-72641-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:38.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.623840+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-72887-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:38.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.626134+0000 mon.a [INF] from='client.? 172.21.15.100:0/2775561011' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-72617-22"}]: dispatch 2024-09-10T18:38:38.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.642319+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-72946-1"}]: dispatch 2024-09-10T18:38:38.995 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.643050+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-72694-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.643364+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.643796+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-4", "overlaypool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.644250+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-72609-16"}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.644539+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-72846-6"}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.645917+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-72601-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:34.646153+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:35.297571+0000 mon.c [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-09-10T18:38:35.307806+0000 mon.a [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (10756 ms) 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (24518 ms total) 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (24519 ms total) 2024-09-10T18:38:38.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-10T18:38:39.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: cluster 2024-09-10T18:38:37.303387+0000 mgr.y (mgr.24512) 171 : cluster [DBG] pgmap v124: 668 pgs: 96 unknown, 88 creating+peering, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T18:38:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.976779+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.987254+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.100:0/3190758578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-72593-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.987477+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.987628+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:38:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.987856+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.988928+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]': finished 2024-09-10T18:38:39.711 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: cluster 2024-09-10T18:38:37.989098+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:38:39.711 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.993871+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:39.711 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.994699+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:38:39.711 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.994980+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:39.711 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.996907+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:39.711 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.997726+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:39.711 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:37.999174+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.027179+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.027471+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.064768+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.099037+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.886626+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.887948+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.895025+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.896145+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.897245+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:39.712 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:39 smithi151 bash[22565]: audit 2024-09-10T18:38:38.897965+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]: dispatch 2024-09-10T18:38:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: cluster 2024-09-10T18:38:37.303387+0000 mgr.y (mgr.24512) 171 : cluster [DBG] pgmap v124: 668 pgs: 96 unknown, 88 creating+peering, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T18:38:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.976779+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.987254+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.100:0/3190758578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-72593-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.987477+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.987628+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:38:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.987856+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.988928+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]': finished 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: cluster 2024-09-10T18:38:37.989098+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.993871+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.994699+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.994980+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.996907+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.997726+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:37.999174+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: cluster 2024-09-10T18:38:37.303387+0000 mgr.y (mgr.24512) 171 : cluster [DBG] pgmap v124: 668 pgs: 96 unknown, 88 creating+peering, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.976779+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.987254+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.100:0/3190758578' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-72593-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.987477+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.987628+0000 mon.a (mon.0) 1019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.987856+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.988928+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-4"}]': finished 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: cluster 2024-09-10T18:38:37.989098+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.993871+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.100:0/2758388437' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.994699+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.994980+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.996907+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.997726+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:37.999174+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.027179+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.027471+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.064768+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.099037+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.886626+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.887948+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.895025+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.896145+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.897245+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[18597]: audit 2024-09-10T18:38:38.897965+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]: dispatch 2024-09-10T18:38:39.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.027179+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.027471+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.064768+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.099037+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.886626+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.887948+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.895025+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.896145+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.897245+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:39.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:39 smithi100 bash[31607]: audit 2024-09-10T18:38:38.897965+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]: dispatch 2024-09-10T18:38:39.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-10T18:38:39.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-10T18:38:39.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-10T18:38:39.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: seed 72887 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (144 ms) 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (11 ms) 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (2 ms) 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (8 ms) 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (166 ms total) 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: 2024-09-10T18:38:39.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1098 ms) 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (5 ms) 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2 ms) 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (17 ms) 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1122 ms total) 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (25403 ms total) 2024-09-10T18:38:39.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-10T18:38:40.254 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-10T18:38:40.254 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:38:40.254 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: seed 72654 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (156 ms) 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (3 ms) 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1145 ms) 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1007 ms) 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-10T18:38:40.255 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (6 ms) 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (5 ms) 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (5 ms) 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2332 ms total) 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: 2024-09-10T18:38:40.256 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1112 ms) 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (7 ms) 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1008 ms) 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1010 ms) 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (7 ms) 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (6 ms) 2024-09-10T18:38:40.257 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (7 ms) 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (5 ms) 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3162 ms total) 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (25920 ms total) 2024-09-10T18:38:40.258 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-10T18:38:40.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: Running main() from gmock_main.cc 2024-09-10T18:38:40.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] Global test environment set-up. 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (133 ms) 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockShared (3 ms) 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1070 ms) 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1043 ms) 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (7 ms) 2024-09-10T18:38:40.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.Unlock (5 ms) 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (7 ms) 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLock (2272 ms total) 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1095 ms) 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (10 ms) 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-10T18:38:40.746 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1011 ms) 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1010 ms) 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (9 ms) 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (6 ms) 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (7 ms) 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (4 ms) 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3152 ms total) 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] Global test environment tear-down 2024-09-10T18:38:40.747 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (26408 ms total) 2024-09-10T18:38:40.748 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: cluster 2024-09-10T18:38:37.909181+0000 osd.4 (osd.4) 5 : cluster [DBG] 17.2 deep-scrub starts 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: cluster 2024-09-10T18:38:37.913718+0000 osd.4 (osd.4) 6 : cluster [DBG] 17.2 deep-scrub ok 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: cluster 2024-09-10T18:38:39.310536+0000 mon.a (mon.0) 1032 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.312595+0000 mon.a (mon.0) 1033 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.313077+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.313451+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.318773+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.100:0/3726578486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.323838+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.323947+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:41.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.324024+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.324372+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.324466+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.324905+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.325204+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]': finished 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: cluster 2024-09-10T18:38:39.325322+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: cluster 2024-09-10T18:38:37.909181+0000 osd.4 (osd.4) 5 : cluster [DBG] 17.2 deep-scrub starts 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: cluster 2024-09-10T18:38:37.913718+0000 osd.4 (osd.4) 6 : cluster [DBG] 17.2 deep-scrub ok 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: cluster 2024-09-10T18:38:39.310536+0000 mon.a (mon.0) 1032 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:38:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.312595+0000 mon.a (mon.0) 1033 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.313077+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.313451+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.318773+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.100:0/3726578486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.323838+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.323947+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.324024+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.324372+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.324466+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.324905+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.325204+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]': finished 2024-09-10T18:38:41.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: cluster 2024-09-10T18:38:39.325322+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.331259+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.331557+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.100:0/891165218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.331891+0000 mon.c (mon.2) 166 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.337893+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.338892+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.344403+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.344751+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.345297+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.349193+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.349517+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:41.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.353840+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.373392+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.378404+0000 mon.a (mon.0) 1050 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.380070+0000 mon.c (mon.2) 168 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: cluster 2024-09-10T18:38:39.807528+0000 mon.a (mon.0) 1051 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.821541+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[31607]: audit 2024-09-10T18:38:39.821759+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.331259+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.331557+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.100:0/891165218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.331891+0000 mon.c (mon.2) 166 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.337893+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.338892+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.344403+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.344751+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.345297+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.349193+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.349517+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.353840+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.373392+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.378404+0000 mon.a (mon.0) 1050 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.380070+0000 mon.c (mon.2) 168 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: cluster 2024-09-10T18:38:39.807528+0000 mon.a (mon.0) 1051 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.821541+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:41.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:40 smithi100 bash[18597]: audit 2024-09-10T18:38:39.821759+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:41.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: cluster 2024-09-10T18:38:37.909181+0000 osd.4 (osd.4) 5 : cluster [DBG] 17.2 deep-scrub starts 2024-09-10T18:38:41.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: cluster 2024-09-10T18:38:37.913718+0000 osd.4 (osd.4) 6 : cluster [DBG] 17.2 deep-scrub ok 2024-09-10T18:38:41.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: cluster 2024-09-10T18:38:39.310536+0000 mon.a (mon.0) 1032 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:38:41.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.312595+0000 mon.a (mon.0) 1033 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.313077+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.100:0/3320234895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.313451+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.318773+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.100:0/3726578486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.323838+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.323947+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.324024+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-4", "tierpool": "test-rados-api-smithi100-72764-4-cache"}]': finished 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.324372+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.324466+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.324905+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.325204+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pg_num": 4}]': finished 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: cluster 2024-09-10T18:38:39.325322+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:38:41.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.331259+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.331557+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.100:0/891165218' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.331891+0000 mon.c (mon.2) 166 : audit [INF] from='client.? 172.21.15.100:0/1527794118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.337893+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.100:0/215148670' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.338892+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.344403+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.344751+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.345297+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.349193+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.349517+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.353840+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.373392+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.378404+0000 mon.a (mon.0) 1050 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.380070+0000 mon.c (mon.2) 168 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:41.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: cluster 2024-09-10T18:38:39.807528+0000 mon.a (mon.0) 1051 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:41.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.821541+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? 172.21.15.100:0/607910459' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-72641-10"}]': finished 2024-09-10T18:38:41.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:40 smithi151 bash[22565]: audit 2024-09-10T18:38:39.821759+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-72887-7"}]': finished 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: cluster 2024-09-10T18:38:39.304529+0000 mgr.y (mgr.24512) 172 : cluster [DBG] pgmap v126: 548 pgs: 96 unknown, 32 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.229959+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.231734+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.100:0/3462894352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.242942+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.243150+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.243344+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.243544+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.243836+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: cluster 2024-09-10T18:38:40.728434+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.730802+0000 mon.a (mon.0) 1060 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.745427+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:41.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.745714+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.813830+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]: dispatch 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.820501+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.820818+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]': finished 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.820979+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: cluster 2024-09-10T18:38:40.821096+0000 mon.a (mon.0) 1066 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.832439+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.100:0/4196550750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.833628+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]: dispatch 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18597]: audit 2024-09-10T18:38:40.834010+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:41.740 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:41] "GET /metrics HTTP/1.1" 200 254242 "" "Prometheus/2.43.0" 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: cluster 2024-09-10T18:38:39.304529+0000 mgr.y (mgr.24512) 172 : cluster [DBG] pgmap v126: 548 pgs: 96 unknown, 32 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.229959+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.231734+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.100:0/3462894352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.242942+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.243150+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.243344+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.243544+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.243836+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:42.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: cluster 2024-09-10T18:38:40.728434+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.730802+0000 mon.a (mon.0) 1060 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.745427+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.745714+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.813830+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]: dispatch 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.820501+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.820818+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]': finished 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.820979+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: cluster 2024-09-10T18:38:40.821096+0000 mon.a (mon.0) 1066 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.832439+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.100:0/4196550750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.833628+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]: dispatch 2024-09-10T18:38:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:41 smithi100 bash[31607]: audit 2024-09-10T18:38:40.834010+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.073 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: cluster 2024-09-10T18:38:39.304529+0000 mgr.y (mgr.24512) 172 : cluster [DBG] pgmap v126: 548 pgs: 96 unknown, 32 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:38:42.073 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.229959+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:42.073 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.231734+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.100:0/3462894352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.242942+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.243150+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.243344+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-72654-10"}]': finished 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.243544+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-72694-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.243836+0000 mon.a (mon.0) 1058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-72795-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: cluster 2024-09-10T18:38:40.728434+0000 mon.a (mon.0) 1059 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.730802+0000 mon.a (mon.0) 1060 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.745427+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.745714+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.813830+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]: dispatch 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.820501+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-72929-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:42.074 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.820818+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]': finished 2024-09-10T18:38:42.075 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.820979+0000 mon.a (mon.0) 1065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-72593-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:42.075 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: cluster 2024-09-10T18:38:40.821096+0000 mon.a (mon.0) 1066 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:38:42.075 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.832439+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.100:0/4196550750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.075 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.833628+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]: dispatch 2024-09-10T18:38:42.075 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:41 smithi151 bash[22565]: audit 2024-09-10T18:38:40.834010+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:42.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:38:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: cluster 2024-09-10T18:38:41.305172+0000 mgr.y (mgr.24512) 173 : cluster [DBG] pgmap v130: 528 pgs: 24 creating+peering, 48 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: audit 2024-09-10T18:38:41.734294+0000 mon.a (mon.0) 1069 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: cluster 2024-09-10T18:38:41.816205+0000 mon.a (mon.0) 1070 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: audit 2024-09-10T18:38:41.823067+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]': finished 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: audit 2024-09-10T18:38:41.823158+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: cluster 2024-09-10T18:38:41.823358+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: audit 2024-09-10T18:38:41.840691+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.100:0/2043860362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:43 smithi151 bash[22565]: audit 2024-09-10T18:38:41.842297+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: cluster 2024-09-10T18:38:41.305172+0000 mgr.y (mgr.24512) 173 : cluster [DBG] pgmap v130: 528 pgs: 24 creating+peering, 48 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: audit 2024-09-10T18:38:41.734294+0000 mon.a (mon.0) 1069 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: cluster 2024-09-10T18:38:41.816205+0000 mon.a (mon.0) 1070 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: audit 2024-09-10T18:38:41.823067+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]': finished 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: audit 2024-09-10T18:38:41.823158+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: cluster 2024-09-10T18:38:41.823358+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: audit 2024-09-10T18:38:41.840691+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.100:0/2043860362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[18597]: audit 2024-09-10T18:38:41.842297+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:43.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: cluster 2024-09-10T18:38:41.305172+0000 mgr.y (mgr.24512) 173 : cluster [DBG] pgmap v130: 528 pgs: 24 creating+peering, 48 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:38:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: audit 2024-09-10T18:38:41.734294+0000 mon.a (mon.0) 1069 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: cluster 2024-09-10T18:38:41.816205+0000 mon.a (mon.0) 1070 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: audit 2024-09-10T18:38:41.823067+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "mode": "writeback"}]': finished 2024-09-10T18:38:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: audit 2024-09-10T18:38:41.823158+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: cluster 2024-09-10T18:38:41.823358+0000 mon.a (mon.0) 1073 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:38:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: audit 2024-09-10T18:38:41.840691+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.100:0/2043860362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:43 smithi100 bash[31607]: audit 2024-09-10T18:38:41.842297+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.676 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:42.072698+0000 mgr.y (mgr.24512) 174 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.164810+0000 mon.a (mon.0) 1075 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.191616+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: cluster 2024-09-10T18:38:43.191937+0000 mon.a (mon.0) 1077 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.201106+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.100:0/1795425399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.204405+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.212397+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.214055+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.100:0/260587237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.220275+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:44 smithi151 bash[22565]: audit 2024-09-10T18:38:43.228834+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:42.072698+0000 mgr.y (mgr.24512) 174 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.164810+0000 mon.a (mon.0) 1075 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.191616+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: cluster 2024-09-10T18:38:43.191937+0000 mon.a (mon.0) 1077 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.201106+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.100:0/1795425399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.204405+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.212397+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.214055+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.100:0/260587237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.220275+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[18597]: audit 2024-09-10T18:38:43.228834+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:42.072698+0000 mgr.y (mgr.24512) 174 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.164810+0000 mon.a (mon.0) 1075 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.191616+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: cluster 2024-09-10T18:38:43.191937+0000 mon.a (mon.0) 1077 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.201106+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.100:0/1795425399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.204405+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.212397+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.214055+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.100:0/260587237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.220275+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:44 smithi100 bash[31607]: audit 2024-09-10T18:38:43.228834+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify_test_cb 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (45 ms) 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-10T18:38:45.115 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93937755903072 err -107 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1009 ms) 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93937755914640 err -107 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1010 ms) 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 322122547200 cookie 93937755935264 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (11 ms) 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 322122547200 cookie 93937755935264 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (7 ms) 2024-09-10T18:38:45.116 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 322122547201 cookie 93937755972016 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (7 ms) 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 322122547200 cookie 93937755956928 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 322122547200 cookie 93937755971952 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (10 ms) 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 322122547202 cookie 93937755971952 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 322122547203 cookie 93937755971952 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3008 ms) 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93937755971952 err -107 2024-09-10T18:38:45.117 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 14922 notify_id 347892350977 cookie 93937755971952 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5017 ms) 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93937755971888 err -107 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1010 ms) 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11134 ms total) 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify_test_cb 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1131 ms) 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1131 ms total) 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: 2024-09-10T18:38:45.118 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-10T18:38:45.119 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (30654 ms total) 2024-09-10T18:38:45.119 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5039 ms) 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (52 ms) 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20027 ms) 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: attempt 0 of 20 2024-09-10T18:38:45.147 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5535 ms) 2024-09-10T18:38:45.148 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30653 ms total) 2024-09-10T18:38:45.148 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: 2024-09-10T18:38:45.148 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-10T18:38:45.148 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30653 ms total) 2024-09-10T18:38:45.148 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-10T18:38:45.196 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (4219 ms) 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (5 ms) 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (4224 ms total) 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-10T18:38:45.197 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (117 ms) 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3864 ms) 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (39 ms) 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (14 ms) 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121473997424 notify_id 373662154754 notifier_gid 25117 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (12 ms) 2024-09-10T18:38:45.198 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485081952 notify_id 373662154754 notifier_gid 25117 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (47 ms) 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485081952 notify_id 373662154755 notifier_gid 25117 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (19 ms) 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485081952 notify_id 373662154753 notifier_gid 25117 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (29 ms) 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485081952 notify_id 373662154754 notifier_gid 25117 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (14 ms) 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485086048 notify_id 373662154755 notifier_gid 25117 2024-09-10T18:38:45.199 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (19 ms) 2024-09-10T18:38:45.200 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-10T18:38:45.200 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: trying... 2024-09-10T18:38:45.200 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485086048 notify_id 373662154755 notifier_gid 25117 2024-09-10T18:38:45.200 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: timed out 2024-09-10T18:38:45.200 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-09-10T18:38:46.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: cluster 2024-09-10T18:38:43.305824+0000 mgr.y (mgr.24512) 175 : cluster [DBG] pgmap v133: 488 pgs: 96 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 op/s 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.377688+0000 mon.a (mon.0) 1081 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.413310+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.413607+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.413728+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: cluster 2024-09-10T18:38:44.413892+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.417256+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? 172.21.15.100:0/687515832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.420476+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.430166+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.431578+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:44.436897+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: cluster 2024-09-10T18:38:45.046578+0000 mon.a (mon.0) 1089 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: cluster 2024-09-10T18:38:45.047447+0000 mon.a (mon.0) 1090 : cluster [WRN] pool 'PoolQuotaPP_smithi100-72601-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: cluster 2024-09-10T18:38:45.049804+0000 mon.a (mon.0) 1091 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:45.064629+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? 172.21.15.100:0/687515832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:45.064761+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:45.064894+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: cluster 2024-09-10T18:38:45.065062+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:45.079907+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.100:0/502607891' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:45.094113+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[18597]: audit 2024-09-10T18:38:45.111434+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.100:0/2152750238' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: cluster 2024-09-10T18:38:43.305824+0000 mgr.y (mgr.24512) 175 : cluster [DBG] pgmap v133: 488 pgs: 96 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 op/s 2024-09-10T18:38:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.377688+0000 mon.a (mon.0) 1081 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.413310+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.413607+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.413728+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: cluster 2024-09-10T18:38:44.413892+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.417256+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? 172.21.15.100:0/687515832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.420476+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.430166+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.431578+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:44.436897+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: cluster 2024-09-10T18:38:45.046578+0000 mon.a (mon.0) 1089 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: cluster 2024-09-10T18:38:45.047447+0000 mon.a (mon.0) 1090 : cluster [WRN] pool 'PoolQuotaPP_smithi100-72601-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: cluster 2024-09-10T18:38:45.049804+0000 mon.a (mon.0) 1091 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:38:46.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:45.064629+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? 172.21.15.100:0/687515832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:45.064761+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:46.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:45.064894+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: cluster 2024-09-10T18:38:45.065062+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:38:46.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:45.079907+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.100:0/502607891' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:45.094113+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:45 smithi100 bash[31607]: audit 2024-09-10T18:38:45.111434+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.100:0/2152750238' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:38:46.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: cluster 2024-09-10T18:38:43.305824+0000 mgr.y (mgr.24512) 175 : cluster [DBG] pgmap v133: 488 pgs: 96 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 op/s 2024-09-10T18:38:46.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.377688+0000 mon.a (mon.0) 1081 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:46.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.413310+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:46.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.413607+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-72593-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.413728+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-72806-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: cluster 2024-09-10T18:38:44.413892+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.417256+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? 172.21.15.100:0/687515832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.420476+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.430166+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.100:0/931246759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.431578+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]: dispatch 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:44.436897+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: cluster 2024-09-10T18:38:45.046578+0000 mon.a (mon.0) 1089 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: cluster 2024-09-10T18:38:45.047447+0000 mon.a (mon.0) 1090 : cluster [WRN] pool 'PoolQuotaPP_smithi100-72601-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: cluster 2024-09-10T18:38:45.049804+0000 mon.a (mon.0) 1091 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:45.064629+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? 172.21.15.100:0/687515832' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:45.064761+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-72929-12"}]': finished 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:45.064894+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: cluster 2024-09-10T18:38:45.065062+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:45.079907+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.100:0/502607891' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:45.094113+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:46.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:45 smithi151 bash[22565]: audit 2024-09-10T18:38:45.111434+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.100:0/2152750238' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:38:47.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:46 smithi151 bash[22565]: audit 2024-09-10T18:38:45.127164+0000 mgr.y (mgr.24512) 176 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:38:47.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:46 smithi151 bash[22565]: cluster 2024-09-10T18:38:45.307949+0000 mgr.y (mgr.24512) 177 : cluster [DBG] pgmap v136: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-09-10T18:38:47.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:46 smithi151 bash[22565]: audit 2024-09-10T18:38:45.674927+0000 mon.a (mon.0) 1097 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:47.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:46 smithi151 bash[22565]: audit 2024-09-10T18:38:46.272163+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:47.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:46 smithi151 bash[22565]: cluster 2024-09-10T18:38:46.272367+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:38:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[18597]: audit 2024-09-10T18:38:45.127164+0000 mgr.y (mgr.24512) 176 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:38:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[18597]: cluster 2024-09-10T18:38:45.307949+0000 mgr.y (mgr.24512) 177 : cluster [DBG] pgmap v136: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-09-10T18:38:47.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[18597]: audit 2024-09-10T18:38:45.674927+0000 mon.a (mon.0) 1097 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:47.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[18597]: audit 2024-09-10T18:38:46.272163+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:47.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[18597]: cluster 2024-09-10T18:38:46.272367+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:38:47.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[31607]: audit 2024-09-10T18:38:45.127164+0000 mgr.y (mgr.24512) 176 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:38:47.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[31607]: cluster 2024-09-10T18:38:45.307949+0000 mgr.y (mgr.24512) 177 : cluster [DBG] pgmap v136: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-09-10T18:38:47.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[31607]: audit 2024-09-10T18:38:45.674927+0000 mon.a (mon.0) 1097 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:47.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[31607]: audit 2024-09-10T18:38:46.272163+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:47.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:46 smithi100 bash[31607]: cluster 2024-09-10T18:38:46.272367+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:38:48.231 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:38:48.231 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] Global test environment set-up. 2024-09-10T18:38:48.231 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-10T18:38:48.231 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-10T18:38:48.231 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (7004 ms) 2024-09-10T18:38:48.231 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-10T18:38:48.231 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.RegisterLate (41 ms) 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: cluster: 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: id: 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: health: HEALTH_WARN 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 8 pool(s) do not have an application enabled 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: services: 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-09-10T18:38:48.232 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: osd: 8 osds: 8 up (since 4m), 8 in (since 5m) 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: foo: 7 portals active (1 hosts, 3 zones) 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: data: 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pools: 16 pools, 452 pgs 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: objects: 209 objects, 584 KiB 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pgs: 7.301% pgs unknown 2024-09-10T18:38:48.233 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 49.336% pgs not active 2024-09-10T18:38:48.234 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 223 creating+peering 2024-09-10T18:38:48.234 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 196 active+clean 2024-09-10T18:38:48.234 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 33 unknown 2024-09-10T18:38:48.234 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.234 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: io: 2024-09-10T18:38:48.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: client: 2.4 KiB/s rd, 2.2 KiB/s wr, 2 op/s rd, 3 op/s wr 2024-09-10T18:38:48.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: cluster: 2024-09-10T18:38:48.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: id: 505ebbde-6fa1-11ef-bcea-c7b262605968 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: health: HEALTH_WARN 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 15 pool(s) do not have an application enabled 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: services: 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: osd: 8 osds: 8 up (since 5m), 8 in (since 5m) 2024-09-10T18:38:48.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: data: 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pools: 29 pools, 868 pgs 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: objects: 233 objects, 585 KiB 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pgs: 55.300% pgs not active 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 480 creating+peering 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 388 active+clean 2024-09-10T18:38:48.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: io: 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: client: 6.9 KiB/s rd, 6.4 KiB/s wr, 9 op/s rd, 8 op/s wr 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.StatusFormat (6224 ms) 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.Status (20468 ms) 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] 4 tests from LibRadosService (33737 ms total) 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] Global test environment tear-down 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [==========] 4 tests from 1 test suite ran. (33737 ms total) 2024-09-10T18:38:48.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ PASSED ] 4 tests. 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:46.270115+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:46.271000+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.100:0/3295390576' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:46.898421+0000 mon.a (mon.0) 1100 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:46.932573+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:46.936871+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:47.268840+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:47.268945+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: cluster 2024-09-10T18:38:47.269108+0000 mon.a (mon.0) 1105 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:38:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: cluster 2024-09-10T18:38:47.309261+0000 mgr.y (mgr.24512) 178 : cluster [DBG] pgmap v139: 552 pgs: 96 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-09-10T18:38:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:47.396802+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:47.412567+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:48 smithi151 bash[22565]: audit 2024-09-10T18:38:48.136281+0000 mon.a (mon.0) 1107 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:46.270115+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:46.271000+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.100:0/3295390576' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:46.898421+0000 mon.a (mon.0) 1100 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:46.932573+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:46.936871+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:47.268840+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:47.268945+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: cluster 2024-09-10T18:38:47.269108+0000 mon.a (mon.0) 1105 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: cluster 2024-09-10T18:38:47.309261+0000 mgr.y (mgr.24512) 178 : cluster [DBG] pgmap v139: 552 pgs: 96 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:47.396802+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:47.412567+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[18597]: audit 2024-09-10T18:38:48.136281+0000 mon.a (mon.0) 1107 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:46.270115+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:46.271000+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.100:0/3295390576' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:46.898421+0000 mon.a (mon.0) 1100 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:46.932573+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:46.936871+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:47.268840+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:47.268945+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-72593-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:48.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: cluster 2024-09-10T18:38:47.269108+0000 mon.a (mon.0) 1105 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:38:48.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: cluster 2024-09-10T18:38:47.309261+0000 mgr.y (mgr.24512) 178 : cluster [DBG] pgmap v139: 552 pgs: 96 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2024-09-10T18:38:48.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:47.396802+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:48.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:47.412567+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:48.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:48 smithi100 bash[31607]: audit 2024-09-10T18:38:48.136281+0000 mon.a (mon.0) 1107 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: cluster 2024-09-10T18:38:47.765893+0000 osd.0 (osd.0) 5 : cluster [DBG] 17.4 deep-scrub starts 2024-09-10T18:38:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: cluster 2024-09-10T18:38:47.905756+0000 osd.0 (osd.0) 6 : cluster [DBG] 17.4 deep-scrub ok 2024-09-10T18:38:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: audit 2024-09-10T18:38:48.279010+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: cluster 2024-09-10T18:38:48.279109+0000 mon.a (mon.0) 1109 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:38:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: audit 2024-09-10T18:38:48.287016+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: audit 2024-09-10T18:38:48.317825+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.100:0/3084918588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: audit 2024-09-10T18:38:48.329897+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: audit 2024-09-10T18:38:48.330857+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[18597]: audit 2024-09-10T18:38:49.712997+0000 mon.a (mon.0) 1112 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: cluster 2024-09-10T18:38:47.765893+0000 osd.0 (osd.0) 5 : cluster [DBG] 17.4 deep-scrub starts 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: cluster 2024-09-10T18:38:47.905756+0000 osd.0 (osd.0) 6 : cluster [DBG] 17.4 deep-scrub ok 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: audit 2024-09-10T18:38:48.279010+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: cluster 2024-09-10T18:38:48.279109+0000 mon.a (mon.0) 1109 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: audit 2024-09-10T18:38:48.287016+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: audit 2024-09-10T18:38:48.317825+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.100:0/3084918588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: audit 2024-09-10T18:38:48.329897+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: audit 2024-09-10T18:38:48.330857+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:50.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:49 smithi100 bash[31607]: audit 2024-09-10T18:38:49.712997+0000 mon.a (mon.0) 1112 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: cluster 2024-09-10T18:38:47.765893+0000 osd.0 (osd.0) 5 : cluster [DBG] 17.4 deep-scrub starts 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: cluster 2024-09-10T18:38:47.905756+0000 osd.0 (osd.0) 6 : cluster [DBG] 17.4 deep-scrub ok 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: audit 2024-09-10T18:38:48.279010+0000 mon.a (mon.0) 1108 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: cluster 2024-09-10T18:38:48.279109+0000 mon.a (mon.0) 1109 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: audit 2024-09-10T18:38:48.287016+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: audit 2024-09-10T18:38:48.317825+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.100:0/3084918588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: audit 2024-09-10T18:38:48.329897+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: audit 2024-09-10T18:38:48.330857+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:50.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:49 smithi151 bash[22565]: audit 2024-09-10T18:38:49.712997+0000 mon.a (mon.0) 1112 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-10T18:38:50.827 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (13229 ms) 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (4579 ms) 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (5694 ms) 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2844 ms) 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-10T18:38:50.828 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3595 ms) 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (2880 ms) 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3498 ms) 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (36319 ms total) 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (36319 ms total) 2024-09-10T18:38:50.829 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-10T18:38:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: cluster 2024-09-10T18:38:49.309974+0000 mgr.y (mgr.24512) 179 : cluster [DBG] pgmap v141: 520 pgs: 64 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.751392+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.100:0/2089089703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.757927+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]': finished 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.758013+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: cluster 2024-09-10T18:38:49.758179+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: cluster 2024-09-10T18:38:49.767933+0000 osd.0 (osd.0) 7 : cluster [DBG] 17.7 deep-scrub starts 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.781451+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.820352+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.100:0/2004618685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.829532+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: cluster 2024-09-10T18:38:49.840601+0000 osd.0 (osd.0) 8 : cluster [DBG] 17.7 deep-scrub ok 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.864670+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.865743+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.866342+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:51.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.877739+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.879817+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.882447+0000 mon.a (mon.0) 1120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.895515+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:49.898200+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: cluster 2024-09-10T18:38:50.051091+0000 mon.a (mon.0) 1122 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[18597]: audit 2024-09-10T18:38:50.715263+0000 mon.a (mon.0) 1123 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: cluster 2024-09-10T18:38:49.309974+0000 mgr.y (mgr.24512) 179 : cluster [DBG] pgmap v141: 520 pgs: 64 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.751392+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.100:0/2089089703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.757927+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]': finished 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.758013+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: cluster 2024-09-10T18:38:49.758179+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:38:51.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: cluster 2024-09-10T18:38:49.767933+0000 osd.0 (osd.0) 7 : cluster [DBG] 17.7 deep-scrub starts 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.781451+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.820352+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.100:0/2004618685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.829532+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: cluster 2024-09-10T18:38:49.840601+0000 osd.0 (osd.0) 8 : cluster [DBG] 17.7 deep-scrub ok 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.864670+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.865743+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.866342+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.877739+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.879817+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.882447+0000 mon.a (mon.0) 1120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.895515+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:49.898200+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: cluster 2024-09-10T18:38:50.051091+0000 mon.a (mon.0) 1122 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:51.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:50 smithi100 bash[31607]: audit 2024-09-10T18:38:50.715263+0000 mon.a (mon.0) 1123 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:51.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: cluster 2024-09-10T18:38:49.309974+0000 mgr.y (mgr.24512) 179 : cluster [DBG] pgmap v141: 520 pgs: 64 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 7.3 KiB/s wr, 8 op/s 2024-09-10T18:38:51.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.751392+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.100:0/2089089703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.757927+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-7"}]': finished 2024-09-10T18:38:51.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.758013+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72996-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: cluster 2024-09-10T18:38:49.758179+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: cluster 2024-09-10T18:38:49.767933+0000 osd.0 (osd.0) 7 : cluster [DBG] 17.7 deep-scrub starts 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.781451+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.820352+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.100:0/2004618685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.829532+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: cluster 2024-09-10T18:38:49.840601+0000 osd.0 (osd.0) 8 : cluster [DBG] 17.7 deep-scrub ok 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.864670+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.865743+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.866342+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.877739+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.879817+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.882447+0000 mon.a (mon.0) 1120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.895515+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:51.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:49.898200+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:38:51.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: cluster 2024-09-10T18:38:50.051091+0000 mon.a (mon.0) 1122 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:51.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:50 smithi151 bash[22565]: audit 2024-09-10T18:38:50.715263+0000 mon.a (mon.0) 1123 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:51.763 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:51] "GET /metrics HTTP/1.1" 200 254242 "" "Prometheus/2.43.0" 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: cluster 2024-09-10T18:38:50.753985+0000 mon.a (mon.0) 1124 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:50.777522+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:50.777710+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:50.777825+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:50.777907+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: cluster 2024-09-10T18:38:50.778026+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:50.783476+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:51.764 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:50.802368+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:50.816045+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:51.441525+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[31607]: audit 2024-09-10T18:38:51.716842+0000 mon.a (mon.0) 1132 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: cluster 2024-09-10T18:38:50.753985+0000 mon.a (mon.0) 1124 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:50.777522+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:50.777710+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:50.777825+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:38:51.765 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:50.777907+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:51.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: cluster 2024-09-10T18:38:50.778026+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:38:51.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:50.783476+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:51.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:50.802368+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:51.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:50.816045+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:51.441525+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:51.766 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:51 smithi100 bash[18597]: audit 2024-09-10T18:38:51.716842+0000 mon.a (mon.0) 1132 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:52.083 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:38:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: cluster 2024-09-10T18:38:50.753985+0000 mon.a (mon.0) 1124 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:50.777522+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:50.777710+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-72593-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:50.777825+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:50.777907+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: cluster 2024-09-10T18:38:50.778026+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:50.783476+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:50.802368+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]: dispatch 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:50.816045+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:51.441525+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:38:52.084 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:51 smithi151 bash[22565]: audit 2024-09-10T18:38:51.716842+0000 mon.a (mon.0) 1132 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:53.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:53 smithi151 bash[22565]: cluster 2024-09-10T18:38:51.311160+0000 mgr.y (mgr.24512) 180 : cluster [DBG] pgmap v144: 488 pgs: 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-09-10T18:38:53.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:53 smithi151 bash[22565]: audit 2024-09-10T18:38:51.782002+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]': finished 2024-09-10T18:38:53.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:53 smithi151 bash[22565]: cluster 2024-09-10T18:38:52.497678+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:38:53.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:53 smithi100 bash[18597]: cluster 2024-09-10T18:38:51.311160+0000 mgr.y (mgr.24512) 180 : cluster [DBG] pgmap v144: 488 pgs: 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-09-10T18:38:53.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:53 smithi100 bash[18597]: audit 2024-09-10T18:38:51.782002+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]': finished 2024-09-10T18:38:53.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:53 smithi100 bash[18597]: cluster 2024-09-10T18:38:52.497678+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:38:53.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:53 smithi100 bash[31607]: cluster 2024-09-10T18:38:51.311160+0000 mgr.y (mgr.24512) 180 : cluster [DBG] pgmap v144: 488 pgs: 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-09-10T18:38:53.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:53 smithi100 bash[31607]: audit 2024-09-10T18:38:51.782002+0000 mon.a (mon.0) 1133 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-7"}]': finished 2024-09-10T18:38:53.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:53 smithi100 bash[31607]: cluster 2024-09-10T18:38:52.497678+0000 mon.a (mon.0) 1134 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:38:54.677 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:38:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:38:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:38:54.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:54 smithi151 bash[22565]: audit 2024-09-10T18:38:52.082140+0000 mgr.y (mgr.24512) 181 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:54.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:54 smithi151 bash[22565]: audit 2024-09-10T18:38:53.044986+0000 mon.a (mon.0) 1135 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:54.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:54 smithi151 bash[22565]: audit 2024-09-10T18:38:53.053733+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:38:54.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:54 smithi151 bash[22565]: cluster 2024-09-10T18:38:53.054547+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:38:54.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:54 smithi151 bash[22565]: audit 2024-09-10T18:38:53.091381+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? 172.21.15.100:0/3786408761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[18597]: audit 2024-09-10T18:38:52.082140+0000 mgr.y (mgr.24512) 181 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[18597]: audit 2024-09-10T18:38:53.044986+0000 mon.a (mon.0) 1135 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[18597]: audit 2024-09-10T18:38:53.053733+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[18597]: cluster 2024-09-10T18:38:53.054547+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[18597]: audit 2024-09-10T18:38:53.091381+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? 172.21.15.100:0/3786408761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[31607]: audit 2024-09-10T18:38:52.082140+0000 mgr.y (mgr.24512) 181 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[31607]: audit 2024-09-10T18:38:53.044986+0000 mon.a (mon.0) 1135 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[31607]: audit 2024-09-10T18:38:53.053733+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-72795-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:38:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[31607]: cluster 2024-09-10T18:38:53.054547+0000 mon.a (mon.0) 1137 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:38:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:54 smithi100 bash[31607]: audit 2024-09-10T18:38:53.091381+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? 172.21.15.100:0/3786408761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: cluster 2024-09-10T18:38:53.311820+0000 mgr.y (mgr.24512) 182 : cluster [DBG] pgmap v147: 432 pgs: 40 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:38:56.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.234235+0000 mon.a (mon.0) 1139 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.246300+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.100:0/1359297348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.253473+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? 172.21.15.100:0/3786408761' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: cluster 2024-09-10T18:38:54.256059+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.260785+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.321434+0000 mon.c (mon.2) 181 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.331945+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.332635+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.335936+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:54.365490+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: cluster 2024-09-10T18:38:55.052471+0000 mon.a (mon.0) 1145 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.064744+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.079707+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:56.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.081667+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.100:0/2120774354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.085189+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.085316+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]': finished 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: cluster 2024-09-10T18:38:55.089488+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.097780+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.098584+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.119292+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.135055+0000 mon.a (mon.0) 1152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:56.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:55 smithi151 bash[22565]: audit 2024-09-10T18:38:55.241554+0000 mon.a (mon.0) 1153 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:56.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: cluster 2024-09-10T18:38:53.311820+0000 mgr.y (mgr.24512) 182 : cluster [DBG] pgmap v147: 432 pgs: 40 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:38:56.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.234235+0000 mon.a (mon.0) 1139 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:56.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.246300+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.100:0/1359297348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.253473+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? 172.21.15.100:0/3786408761' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: cluster 2024-09-10T18:38:54.256059+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.260785+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.321434+0000 mon.c (mon.2) 181 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.331945+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]: dispatch 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.332635+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.335936+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:54.365490+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]: dispatch 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: cluster 2024-09-10T18:38:55.052471+0000 mon.a (mon.0) 1145 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.064744+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.079707+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.081667+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.100:0/2120774354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.085189+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.085316+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]': finished 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: cluster 2024-09-10T18:38:55.089488+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.097780+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.098584+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.119292+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.135055+0000 mon.a (mon.0) 1152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[18597]: audit 2024-09-10T18:38:55.241554+0000 mon.a (mon.0) 1153 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: cluster 2024-09-10T18:38:53.311820+0000 mgr.y (mgr.24512) 182 : cluster [DBG] pgmap v147: 432 pgs: 40 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.234235+0000 mon.a (mon.0) 1139 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.246300+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.100:0/1359297348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.253473+0000 mon.a (mon.0) 1140 : audit [INF] from='client.? 172.21.15.100:0/3786408761' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: cluster 2024-09-10T18:38:54.256059+0000 mon.a (mon.0) 1141 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.260785+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.321434+0000 mon.c (mon.2) 181 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.331945+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.332635+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.335936+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:54.365490+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]: dispatch 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: cluster 2024-09-10T18:38:55.052471+0000 mon.a (mon.0) 1145 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:56.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.064744+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-72593-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.079707+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.081667+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.100:0/2120774354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.085189+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.085316+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4"}]': finished 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: cluster 2024-09-10T18:38:55.089488+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.097780+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]: dispatch 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.098584+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.119292+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.135055+0000 mon.a (mon.0) 1152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:38:56.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:55 smithi100 bash[31607]: audit 2024-09-10T18:38:55.241554+0000 mon.a (mon.0) 1153 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: cluster 2024-09-10T18:38:55.313837+0000 mgr.y (mgr.24512) 183 : cluster [DBG] pgmap v150: 464 pgs: 32 creating+peering, 432 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.057447+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: cluster 2024-09-10T18:38:56.058897+0000 mon.a (mon.0) 1154 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.061589+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.064268+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]': finished 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.064349+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.064444+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: cluster 2024-09-10T18:38:56.064598+0000 mon.a (mon.0) 1158 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:38:57.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.088998+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:38:57.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.095193+0000 mon.c (mon.2) 182 : audit [INF] from='client.? 172.21.15.100:0/278582160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.101803+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.736869+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:57 smithi151 bash[22565]: audit 2024-09-10T18:38:56.737119+0000 mon.a (mon.0) 1162 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: cluster 2024-09-10T18:38:55.313837+0000 mgr.y (mgr.24512) 183 : cluster [DBG] pgmap v150: 464 pgs: 32 creating+peering, 432 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.057447+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: cluster 2024-09-10T18:38:56.058897+0000 mon.a (mon.0) 1154 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.061589+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.064268+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]': finished 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.064349+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.064444+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: cluster 2024-09-10T18:38:56.064598+0000 mon.a (mon.0) 1158 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.088998+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.095193+0000 mon.c (mon.2) 182 : audit [INF] from='client.? 172.21.15.100:0/278582160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.101803+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.736869+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[18597]: audit 2024-09-10T18:38:56.737119+0000 mon.a (mon.0) 1162 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: cluster 2024-09-10T18:38:55.313837+0000 mgr.y (mgr.24512) 183 : cluster [DBG] pgmap v150: 464 pgs: 32 creating+peering, 432 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.057447+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: cluster 2024-09-10T18:38:56.058897+0000 mon.a (mon.0) 1154 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.061589+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.100:0/3875370480' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.064268+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6"}]': finished 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.064349+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.064444+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: cluster 2024-09-10T18:38:56.064598+0000 mon.a (mon.0) 1158 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.088998+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:38:57.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.095193+0000 mon.c (mon.2) 182 : audit [INF] from='client.? 172.21.15.100:0/278582160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.101803+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.736869+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:57.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:57 smithi100 bash[31607]: audit 2024-09-10T18:38:56.737119+0000 mon.a (mon.0) 1162 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:58.409 INFO:tasks.workunit.client.0.smithi100.stdout:api_watch_notify_pp: flushed 2024-09-10T18:38:58.409 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3028 ms) 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: trying... 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485086048 notify_id 386547056642 notifier_gid 25117 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: timed out 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushed 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3030 ms) 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: List watches 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485086048 notify_id 395136991236 notifier_gid 25117 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 done 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: watch_check 2024-09-10T18:38:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: unwatch2 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: done 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3017 ms) 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: List watches 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94121485086048 notify_id 408021893124 notifier_gid 25117 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 done 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: watch_check 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: unwatch2 2024-09-10T18:38:58.411 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-09-10T18:38:58.412 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: done 2024-09-10T18:38:58.412 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3100 ms) 2024-09-10T18:38:58.412 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16349 ms total) 2024-09-10T18:38:58.412 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: 2024-09-10T18:38:58.412 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-10T18:38:58.412 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (43941 ms total) 2024-09-10T18:38:58.412 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: cluster 2024-09-10T18:38:57.314501+0000 mgr.y (mgr.24512) 184 : cluster [DBG] pgmap v152: 464 pgs: 32 unknown, 32 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.368528+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]: dispatch 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.370192+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]': finished 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.370298+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.370411+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: cluster 2024-09-10T18:38:57.370574+0000 mon.a (mon.0) 1166 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:38:58.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.374110+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.100:0/2791466749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:58.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.391179+0000 mon.a (mon.0) 1167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]: dispatch 2024-09-10T18:38:58.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.430309+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:58.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:58 smithi151 bash[22565]: audit 2024-09-10T18:38:57.749541+0000 mon.a (mon.0) 1169 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: cluster 2024-09-10T18:38:57.314501+0000 mgr.y (mgr.24512) 184 : cluster [DBG] pgmap v152: 464 pgs: 32 unknown, 32 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.368528+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]: dispatch 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.370192+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]': finished 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.370298+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.370411+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: cluster 2024-09-10T18:38:57.370574+0000 mon.a (mon.0) 1166 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.374110+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.100:0/2791466749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.391179+0000 mon.a (mon.0) 1167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]: dispatch 2024-09-10T18:38:58.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.430309+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[18597]: audit 2024-09-10T18:38:57.749541+0000 mon.a (mon.0) 1169 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: cluster 2024-09-10T18:38:57.314501+0000 mgr.y (mgr.24512) 184 : cluster [DBG] pgmap v152: 464 pgs: 32 unknown, 32 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 3.5 KiB/s wr, 6 op/s 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.368528+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]: dispatch 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.370192+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-9"}]': finished 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.370298+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-72946-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.370411+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-72601-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: cluster 2024-09-10T18:38:57.370574+0000 mon.a (mon.0) 1166 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:38:58.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.374110+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.100:0/2791466749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:58.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.391179+0000 mon.a (mon.0) 1167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]: dispatch 2024-09-10T18:38:58.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.430309+0000 mon.a (mon.0) 1168 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:38:58.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:58 smithi100 bash[31607]: audit 2024-09-10T18:38:57.749541+0000 mon.a (mon.0) 1169 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:59.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:59 smithi151 bash[22565]: cluster 2024-09-10T18:38:58.366775+0000 mon.a (mon.0) 1170 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:59.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:59 smithi151 bash[22565]: audit 2024-09-10T18:38:58.382944+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]': finished 2024-09-10T18:38:59.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:59 smithi151 bash[22565]: audit 2024-09-10T18:38:58.383120+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:59.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:59 smithi151 bash[22565]: cluster 2024-09-10T18:38:58.383267+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:38:59.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:59 smithi151 bash[22565]: audit 2024-09-10T18:38:58.574811+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:59.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:59 smithi151 bash[22565]: audit 2024-09-10T18:38:58.592052+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:59.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:38:59 smithi151 bash[22565]: audit 2024-09-10T18:38:59.193457+0000 mon.a (mon.0) 1175 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[18597]: cluster 2024-09-10T18:38:58.366775+0000 mon.a (mon.0) 1170 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[18597]: audit 2024-09-10T18:38:58.382944+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]': finished 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[18597]: audit 2024-09-10T18:38:58.383120+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[18597]: cluster 2024-09-10T18:38:58.383267+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[18597]: audit 2024-09-10T18:38:58.574811+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[18597]: audit 2024-09-10T18:38:58.592052+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[18597]: audit 2024-09-10T18:38:59.193457+0000 mon.a (mon.0) 1175 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:38:59.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[31607]: cluster 2024-09-10T18:38:58.366775+0000 mon.a (mon.0) 1170 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:38:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[31607]: audit 2024-09-10T18:38:58.382944+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-9", "mode": "writeback"}]': finished 2024-09-10T18:38:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[31607]: audit 2024-09-10T18:38:58.383120+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-72593-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:38:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[31607]: cluster 2024-09-10T18:38:58.383267+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:38:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[31607]: audit 2024-09-10T18:38:58.574811+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[31607]: audit 2024-09-10T18:38:58.592052+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:38:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:38:59 smithi100 bash[31607]: audit 2024-09-10T18:38:59.193457+0000 mon.a (mon.0) 1175 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: cluster 2024-09-10T18:38:59.315102+0000 mgr.y (mgr.24512) 185 : cluster [DBG] pgmap v155: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 241 B/s wr, 0 op/s 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:38:59.390773+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:38:59.399442+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: cluster 2024-09-10T18:38:59.399546+0000 mon.a (mon.0) 1177 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:38:59.405562+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.100:0/2176889734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:38:59.406729+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:38:59.406999+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.204826+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:39:01.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.205956+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.206915+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.207840+0000 mon.c (mon.2) 187 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.208740+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.209855+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.210929+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.212042+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.213134+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:39:01.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: cluster 2024-09-10T18:38:59.315102+0000 mgr.y (mgr.24512) 185 : cluster [DBG] pgmap v155: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 241 B/s wr, 0 op/s 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:38:59.390773+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:38:59.399442+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: cluster 2024-09-10T18:38:59.399546+0000 mon.a (mon.0) 1177 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:38:59.405562+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.100:0/2176889734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:38:59.406729+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:38:59.406999+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.204826+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.205956+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.206915+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.207840+0000 mon.c (mon.2) 187 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.208740+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:39:01.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.209855+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.210929+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.212042+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.213134+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.214188+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: cluster 2024-09-10T18:39:00.262104+0000 mon.a (mon.0) 1180 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: cluster 2024-09-10T18:39:00.264296+0000 mon.a (mon.0) 1181 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.268243+0000 mon.a (mon.0) 1182 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.278982+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.279078+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]': finished 2024-09-10T18:39:01.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: cluster 2024-09-10T18:39:00.279274+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[18597]: audit 2024-09-10T18:39:00.300291+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.214188+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: cluster 2024-09-10T18:39:00.262104+0000 mon.a (mon.0) 1180 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: cluster 2024-09-10T18:39:00.264296+0000 mon.a (mon.0) 1181 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.268243+0000 mon.a (mon.0) 1182 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.278982+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.279078+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]': finished 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: cluster 2024-09-10T18:39:00.279274+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:39:01.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:00 smithi100 bash[31607]: audit 2024-09-10T18:39:00.300291+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: cluster 2024-09-10T18:38:59.315102+0000 mgr.y (mgr.24512) 185 : cluster [DBG] pgmap v155: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 241 B/s wr, 0 op/s 2024-09-10T18:39:01.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:38:59.390773+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:38:59.399442+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: cluster 2024-09-10T18:38:59.399546+0000 mon.a (mon.0) 1177 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:38:59.405562+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.100:0/2176889734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:38:59.406729+0000 mon.a (mon.0) 1178 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:38:59.406999+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.204826+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.205956+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.206915+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.207840+0000 mon.c (mon.2) 187 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.208740+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.209855+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:39:01.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.210929+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.212042+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.213134+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.214188+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: cluster 2024-09-10T18:39:00.262104+0000 mon.a (mon.0) 1180 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: cluster 2024-09-10T18:39:00.264296+0000 mon.a (mon.0) 1181 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.268243+0000 mon.a (mon.0) 1182 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.278982+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-72601-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.279078+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-9"}]': finished 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: cluster 2024-09-10T18:39:00.279274+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:39:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:00 smithi151 bash[22565]: audit 2024-09-10T18:39:00.300291+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:01.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-10T18:39:01.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] Global test environment set-up. 2024-09-10T18:39:01.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-09-10T18:38:14.385+0000 7f9c0fc9f6c0 -1 failed for service _ceph-mon._tcp 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-09-10T18:38:14.385+0000 7f9c0fc9f6c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (41 ms) 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5011 ms) 2024-09-10T18:39:01.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5052 ms total) 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x564928c654b0 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x564928c69cb0 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 2 aios 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x564928c654b0 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x564928c69cb0 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: done. 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (12447 ms) 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (12447 ms total) 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-09-10T18:39:01.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (1 ms) 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.Exec (4 ms) 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (6 ms) 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (4 ms) 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.Applications (6497 ms) 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-10T18:39:01.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-10T18:39:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-10T18:39:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (20747 ms) 2024-09-10T18:39:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 8 tests from LibRadosMisc (27260 ms total) 2024-09-10T18:39:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-09-10T18:39:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] Global test environment tear-down 2024-09-10T18:39:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (46948 ms total) 2024-09-10T18:39:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-10T18:39:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:39:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:01] "GET /metrics HTTP/1.1" 200 249693 "" "Prometheus/2.43.0" 2024-09-10T18:39:02.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:39:02.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.205152+0000 mgr.y (mgr.24512) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:39:02.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.206177+0000 mgr.y (mgr.24512) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:39:02.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.207132+0000 mgr.y (mgr.24512) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:39:02.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.208044+0000 mgr.y (mgr.24512) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:39:02.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.208947+0000 mgr.y (mgr.24512) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.210118+0000 mgr.y (mgr.24512) 191 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.211213+0000 mgr.y (mgr.24512) 192 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.212343+0000 mgr.y (mgr.24512) 193 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.213435+0000 mgr.y (mgr.24512) 194 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:00.214469+0000 mgr.y (mgr.24512) 195 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:01.271319+0000 mon.a (mon.0) 1187 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:01.287102+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: cluster 2024-09-10T18:39:01.287570+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:39:02.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:02 smithi151 bash[22565]: audit 2024-09-10T18:39:01.298603+0000 mon.a (mon.0) 1190 : audit [INF] from='client.? 172.21.15.100:0/1586070902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-72593-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.205152+0000 mgr.y (mgr.24512) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.206177+0000 mgr.y (mgr.24512) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.207132+0000 mgr.y (mgr.24512) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.208044+0000 mgr.y (mgr.24512) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.208947+0000 mgr.y (mgr.24512) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.210118+0000 mgr.y (mgr.24512) 191 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.211213+0000 mgr.y (mgr.24512) 192 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.212343+0000 mgr.y (mgr.24512) 193 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:39:02.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.213435+0000 mgr.y (mgr.24512) 194 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:00.214469+0000 mgr.y (mgr.24512) 195 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:01.271319+0000 mon.a (mon.0) 1187 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:01.287102+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: cluster 2024-09-10T18:39:01.287570+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[31607]: audit 2024-09-10T18:39:01.298603+0000 mon.a (mon.0) 1190 : audit [INF] from='client.? 172.21.15.100:0/1586070902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-72593-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.205152+0000 mgr.y (mgr.24512) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.206177+0000 mgr.y (mgr.24512) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.207132+0000 mgr.y (mgr.24512) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.208044+0000 mgr.y (mgr.24512) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.208947+0000 mgr.y (mgr.24512) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:39:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.210118+0000 mgr.y (mgr.24512) 191 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.211213+0000 mgr.y (mgr.24512) 192 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.212343+0000 mgr.y (mgr.24512) 193 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.213435+0000 mgr.y (mgr.24512) 194 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:00.214469+0000 mgr.y (mgr.24512) 195 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:01.271319+0000 mon.a (mon.0) 1187 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:01.287102+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72779-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: cluster 2024-09-10T18:39:01.287570+0000 mon.a (mon.0) 1189 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:39:02.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:02 smithi100 bash[18597]: audit 2024-09-10T18:39:01.298603+0000 mon.a (mon.0) 1190 : audit [INF] from='client.? 172.21.15.100:0/1586070902' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-72593-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: cluster 2024-09-10T18:39:01.193791+0000 osd.6 (osd.6) 7 : cluster [DBG] 17.6 deep-scrub starts 2024-09-10T18:39:03.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: cluster 2024-09-10T18:39:01.248606+0000 osd.6 (osd.6) 8 : cluster [DBG] 17.6 deep-scrub ok 2024-09-10T18:39:03.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: cluster 2024-09-10T18:39:01.315798+0000 mgr.y (mgr.24512) 196 : cluster [DBG] pgmap v159: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:39:03.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: cluster 2024-09-10T18:39:01.641579+0000 osd.4 (osd.4) 7 : cluster [DBG] 17.3 deep-scrub starts 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: cluster 2024-09-10T18:39:01.644869+0000 osd.4 (osd.4) 8 : cluster [DBG] 17.3 deep-scrub ok 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.090055+0000 mgr.y (mgr.24512) 197 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.544128+0000 mon.a (mon.0) 1191 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.544550+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.556226+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? 172.21.15.100:0/1586070902' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-72593-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: cluster 2024-09-10T18:39:02.556994+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.560060+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.100:0/3432162580' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.561767+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.562538+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:03 smithi151 bash[22565]: audit 2024-09-10T18:39:02.562985+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:39:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: cluster 2024-09-10T18:39:01.193791+0000 osd.6 (osd.6) 7 : cluster [DBG] 17.6 deep-scrub starts 2024-09-10T18:39:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: cluster 2024-09-10T18:39:01.248606+0000 osd.6 (osd.6) 8 : cluster [DBG] 17.6 deep-scrub ok 2024-09-10T18:39:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: cluster 2024-09-10T18:39:01.315798+0000 mgr.y (mgr.24512) 196 : cluster [DBG] pgmap v159: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:39:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: cluster 2024-09-10T18:39:01.641579+0000 osd.4 (osd.4) 7 : cluster [DBG] 17.3 deep-scrub starts 2024-09-10T18:39:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: cluster 2024-09-10T18:39:01.644869+0000 osd.4 (osd.4) 8 : cluster [DBG] 17.3 deep-scrub ok 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.090055+0000 mgr.y (mgr.24512) 197 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.544128+0000 mon.a (mon.0) 1191 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.544550+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.556226+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? 172.21.15.100:0/1586070902' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-72593-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: cluster 2024-09-10T18:39:02.556994+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.560060+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.100:0/3432162580' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.561767+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.562538+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[18597]: audit 2024-09-10T18:39:02.562985+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:39:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: cluster 2024-09-10T18:39:01.193791+0000 osd.6 (osd.6) 7 : cluster [DBG] 17.6 deep-scrub starts 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: cluster 2024-09-10T18:39:01.248606+0000 osd.6 (osd.6) 8 : cluster [DBG] 17.6 deep-scrub ok 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: cluster 2024-09-10T18:39:01.315798+0000 mgr.y (mgr.24512) 196 : cluster [DBG] pgmap v159: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: cluster 2024-09-10T18:39:01.641579+0000 osd.4 (osd.4) 7 : cluster [DBG] 17.3 deep-scrub starts 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: cluster 2024-09-10T18:39:01.644869+0000 osd.4 (osd.4) 8 : cluster [DBG] 17.3 deep-scrub ok 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.090055+0000 mgr.y (mgr.24512) 197 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.544128+0000 mon.a (mon.0) 1191 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.544550+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.556226+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? 172.21.15.100:0/1586070902' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-72593-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: cluster 2024-09-10T18:39:02.556994+0000 mon.a (mon.0) 1193 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:39:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.560060+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.100:0/3432162580' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.561767+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.562538+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:03.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:03 smithi100 bash[31607]: audit 2024-09-10T18:39:02.562985+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:39:04.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: cluster 2024-09-10T18:39:02.210588+0000 osd.6 (osd.6) 9 : cluster [DBG] 17.5 deep-scrub starts 2024-09-10T18:39:04.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: cluster 2024-09-10T18:39:02.214734+0000 osd.6 (osd.6) 10 : cluster [DBG] 17.5 deep-scrub ok 2024-09-10T18:39:04.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: cluster 2024-09-10T18:39:02.522584+0000 osd.3 (osd.3) 5 : cluster [DBG] 17.9 deep-scrub starts 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: cluster 2024-09-10T18:39:02.525822+0000 osd.3 (osd.3) 6 : cluster [DBG] 17.9 deep-scrub ok 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: cluster 2024-09-10T18:39:03.317340+0000 mgr.y (mgr.24512) 198 : cluster [DBG] pgmap v161: 460 pgs: 160 unknown, 300 active+clean; 217 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: audit 2024-09-10T18:39:03.547687+0000 mon.a (mon.0) 1197 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: audit 2024-09-10T18:39:03.558076+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: audit 2024-09-10T18:39:03.558315+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: audit 2024-09-10T18:39:03.558531+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: cluster 2024-09-10T18:39:03.558910+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: audit 2024-09-10T18:39:03.562342+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: audit 2024-09-10T18:39:03.567306+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[22565]: audit 2024-09-10T18:39:03.568053+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72779-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: cluster 2024-09-10T18:39:02.210588+0000 osd.6 (osd.6) 9 : cluster [DBG] 17.5 deep-scrub starts 2024-09-10T18:39:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: cluster 2024-09-10T18:39:02.214734+0000 osd.6 (osd.6) 10 : cluster [DBG] 17.5 deep-scrub ok 2024-09-10T18:39:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: cluster 2024-09-10T18:39:02.522584+0000 osd.3 (osd.3) 5 : cluster [DBG] 17.9 deep-scrub starts 2024-09-10T18:39:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: cluster 2024-09-10T18:39:02.525822+0000 osd.3 (osd.3) 6 : cluster [DBG] 17.9 deep-scrub ok 2024-09-10T18:39:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: cluster 2024-09-10T18:39:03.317340+0000 mgr.y (mgr.24512) 198 : cluster [DBG] pgmap v161: 460 pgs: 160 unknown, 300 active+clean; 217 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: audit 2024-09-10T18:39:03.547687+0000 mon.a (mon.0) 1197 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: audit 2024-09-10T18:39:03.558076+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: audit 2024-09-10T18:39:03.558315+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: audit 2024-09-10T18:39:03.558531+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: cluster 2024-09-10T18:39:03.558910+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: audit 2024-09-10T18:39:03.562342+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: audit 2024-09-10T18:39:03.567306+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[18597]: audit 2024-09-10T18:39:03.568053+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72779-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: cluster 2024-09-10T18:39:02.210588+0000 osd.6 (osd.6) 9 : cluster [DBG] 17.5 deep-scrub starts 2024-09-10T18:39:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: cluster 2024-09-10T18:39:02.214734+0000 osd.6 (osd.6) 10 : cluster [DBG] 17.5 deep-scrub ok 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: cluster 2024-09-10T18:39:02.522584+0000 osd.3 (osd.3) 5 : cluster [DBG] 17.9 deep-scrub starts 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: cluster 2024-09-10T18:39:02.525822+0000 osd.3 (osd.3) 6 : cluster [DBG] 17.9 deep-scrub ok 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: cluster 2024-09-10T18:39:03.317340+0000 mgr.y (mgr.24512) 198 : cluster [DBG] pgmap v161: 460 pgs: 160 unknown, 300 active+clean; 217 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: audit 2024-09-10T18:39:03.547687+0000 mon.a (mon.0) 1197 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: audit 2024-09-10T18:39:03.558076+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-72601-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: audit 2024-09-10T18:39:03.558315+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: audit 2024-09-10T18:39:03.558531+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: cluster 2024-09-10T18:39:03.558910+0000 mon.a (mon.0) 1201 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: audit 2024-09-10T18:39:03.562342+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: audit 2024-09-10T18:39:03.567306+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:04.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:04 smithi100 bash[31607]: audit 2024-09-10T18:39:03.568053+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72779-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:04.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:39:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:05.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: cluster 2024-09-10T18:39:04.216011+0000 osd.7 (osd.7) 7 : cluster [DBG] 17.0 deep-scrub starts 2024-09-10T18:39:05.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: cluster 2024-09-10T18:39:04.320613+0000 osd.7 (osd.7) 8 : cluster [DBG] 17.0 deep-scrub ok 2024-09-10T18:39:05.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.549514+0000 mon.a (mon.0) 1204 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:05.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.560932+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.561203+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72779-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: cluster 2024-09-10T18:39:04.561514+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.567236+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache"}]: dispatch 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.570193+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.570997+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.100:0/1551904739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.572779+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.573366+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.583299+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:05.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:05 smithi151 bash[22565]: audit 2024-09-10T18:39:04.599035+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:05.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: cluster 2024-09-10T18:39:04.216011+0000 osd.7 (osd.7) 7 : cluster [DBG] 17.0 deep-scrub starts 2024-09-10T18:39:05.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: cluster 2024-09-10T18:39:04.320613+0000 osd.7 (osd.7) 8 : cluster [DBG] 17.0 deep-scrub ok 2024-09-10T18:39:05.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.549514+0000 mon.a (mon.0) 1204 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:05.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.560932+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.561203+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72779-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: cluster 2024-09-10T18:39:04.561514+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.567236+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache"}]: dispatch 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.570193+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.570997+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.100:0/1551904739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.572779+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.573366+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.583299+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[18597]: audit 2024-09-10T18:39:04.599035+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:05.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: cluster 2024-09-10T18:39:04.216011+0000 osd.7 (osd.7) 7 : cluster [DBG] 17.0 deep-scrub starts 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: cluster 2024-09-10T18:39:04.320613+0000 osd.7 (osd.7) 8 : cluster [DBG] 17.0 deep-scrub ok 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.549514+0000 mon.a (mon.0) 1204 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.560932+0000 mon.a (mon.0) 1205 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.561203+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72779-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: cluster 2024-09-10T18:39:04.561514+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.567236+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache"}]: dispatch 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.570193+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.100:0/1650495186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.570997+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.100:0/1551904739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.572779+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]: dispatch 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.573366+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.583299+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:05 smithi100 bash[31607]: audit 2024-09-10T18:39:04.599035+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:06.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: cluster 2024-09-10T18:39:04.776405+0000 osd.0 (osd.0) 9 : cluster [DBG] 17.1 deep-scrub starts 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: cluster 2024-09-10T18:39:04.784371+0000 osd.0 (osd.0) 10 : cluster [DBG] 17.1 deep-scrub ok 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.248589+0000 osd.7 (osd.7) 9 : cluster [DBG] 17.8 deep-scrub starts 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.251956+0000 osd.7 (osd.7) 10 : cluster [DBG] 17.8 deep-scrub ok 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.265710+0000 mon.a (mon.0) 1212 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.320626+0000 mgr.y (mgr.24512) 199 : cluster [DBG] pgmap v164: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1015 B/s rd, 255 KiB/s wr, 2 op/s 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.551206+0000 mon.a (mon.0) 1213 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.555628+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.566887+0000 mon.a (mon.0) 1214 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache"}]': finished 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.567089+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.567227+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.567419+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:39:06.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.567527+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:39:06.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.576132+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.100:0/4233468715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:06.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.577540+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:06.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.586020+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:06.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.608604+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:06.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.609983+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:06.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:06 smithi151 bash[22565]: audit 2024-09-10T18:39:05.610348+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:06.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: cluster 2024-09-10T18:39:04.776405+0000 osd.0 (osd.0) 9 : cluster [DBG] 17.1 deep-scrub starts 2024-09-10T18:39:06.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: cluster 2024-09-10T18:39:04.784371+0000 osd.0 (osd.0) 10 : cluster [DBG] 17.1 deep-scrub ok 2024-09-10T18:39:06.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.248589+0000 osd.7 (osd.7) 9 : cluster [DBG] 17.8 deep-scrub starts 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.251956+0000 osd.7 (osd.7) 10 : cluster [DBG] 17.8 deep-scrub ok 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.265710+0000 mon.a (mon.0) 1212 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.320626+0000 mgr.y (mgr.24512) 199 : cluster [DBG] pgmap v164: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1015 B/s rd, 255 KiB/s wr, 2 op/s 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.551206+0000 mon.a (mon.0) 1213 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.555628+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.566887+0000 mon.a (mon.0) 1214 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache"}]': finished 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.567089+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.567227+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.567419+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.567527+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.576132+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.100:0/4233468715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.577540+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.586020+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.608604+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: cluster 2024-09-10T18:39:04.776405+0000 osd.0 (osd.0) 9 : cluster [DBG] 17.1 deep-scrub starts 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: cluster 2024-09-10T18:39:04.784371+0000 osd.0 (osd.0) 10 : cluster [DBG] 17.1 deep-scrub ok 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.248589+0000 osd.7 (osd.7) 9 : cluster [DBG] 17.8 deep-scrub starts 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.251956+0000 osd.7 (osd.7) 10 : cluster [DBG] 17.8 deep-scrub ok 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.265710+0000 mon.a (mon.0) 1212 : cluster [WRN] Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.320626+0000 mgr.y (mgr.24512) 199 : cluster [DBG] pgmap v164: 420 pgs: 32 creating+peering, 388 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1015 B/s rd, 255 KiB/s wr, 2 op/s 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.551206+0000 mon.a (mon.0) 1213 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.555628+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.566887+0000 mon.a (mon.0) 1214 : audit [INF] from='client.? 172.21.15.100:0/4019859257' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72779-10", "tierpool":"test-rados-api-smithi100-72779-10-cache"}]': finished 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.567089+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-72795-9"}]': finished 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.567227+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-72593-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.567419+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.567527+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.576132+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.100:0/4233468715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.577540+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.586020+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.608604+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:06.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.609983+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:06.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[18597]: audit 2024-09-10T18:39:05.610348+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:06.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.609983+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:06.547 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:06 smithi100 bash[31607]: audit 2024-09-10T18:39:05.610348+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:07.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.567846+0000 osd.4 (osd.4) 9 : cluster [DBG] 17.2 deep-scrub starts 2024-09-10T18:39:07.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.587469+0000 osd.4 (osd.4) 10 : cluster [DBG] 17.2 deep-scrub ok 2024-09-10T18:39:07.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.764884+0000 osd.0 (osd.0) 11 : cluster [DBG] 17.4 deep-scrub starts 2024-09-10T18:39:07.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: cluster 2024-09-10T18:39:05.861685+0000 osd.0 (osd.0) 12 : cluster [DBG] 17.4 deep-scrub ok 2024-09-10T18:39:07.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: audit 2024-09-10T18:39:06.552869+0000 mon.a (mon.0) 1224 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:07.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: audit 2024-09-10T18:39:06.558435+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]: dispatch 2024-09-10T18:39:07.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: audit 2024-09-10T18:39:06.571419+0000 mon.a (mon.0) 1225 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:07.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: audit 2024-09-10T18:39:06.571762+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]': finished 2024-09-10T18:39:07.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: audit 2024-09-10T18:39:06.572239+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:07.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: cluster 2024-09-10T18:39:06.572619+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:39:07.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: audit 2024-09-10T18:39:06.576170+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:07.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:07 smithi151 bash[22565]: audit 2024-09-10T18:39:06.577276+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]: dispatch 2024-09-10T18:39:07.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.567846+0000 osd.4 (osd.4) 9 : cluster [DBG] 17.2 deep-scrub starts 2024-09-10T18:39:07.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.587469+0000 osd.4 (osd.4) 10 : cluster [DBG] 17.2 deep-scrub ok 2024-09-10T18:39:07.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.764884+0000 osd.0 (osd.0) 11 : cluster [DBG] 17.4 deep-scrub starts 2024-09-10T18:39:07.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: cluster 2024-09-10T18:39:05.861685+0000 osd.0 (osd.0) 12 : cluster [DBG] 17.4 deep-scrub ok 2024-09-10T18:39:07.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: audit 2024-09-10T18:39:06.552869+0000 mon.a (mon.0) 1224 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: audit 2024-09-10T18:39:06.558435+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]: dispatch 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: audit 2024-09-10T18:39:06.571419+0000 mon.a (mon.0) 1225 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: audit 2024-09-10T18:39:06.571762+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]': finished 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: audit 2024-09-10T18:39:06.572239+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: cluster 2024-09-10T18:39:06.572619+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: audit 2024-09-10T18:39:06.576170+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[18597]: audit 2024-09-10T18:39:06.577276+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]: dispatch 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.567846+0000 osd.4 (osd.4) 9 : cluster [DBG] 17.2 deep-scrub starts 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.587469+0000 osd.4 (osd.4) 10 : cluster [DBG] 17.2 deep-scrub ok 2024-09-10T18:39:07.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.764884+0000 osd.0 (osd.0) 11 : cluster [DBG] 17.4 deep-scrub starts 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: cluster 2024-09-10T18:39:05.861685+0000 osd.0 (osd.0) 12 : cluster [DBG] 17.4 deep-scrub ok 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: audit 2024-09-10T18:39:06.552869+0000 mon.a (mon.0) 1224 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: audit 2024-09-10T18:39:06.558435+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]: dispatch 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: audit 2024-09-10T18:39:06.571419+0000 mon.a (mon.0) 1225 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-72601-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: audit 2024-09-10T18:39:06.571762+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-11"}]': finished 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: audit 2024-09-10T18:39:06.572239+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: cluster 2024-09-10T18:39:06.572619+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: audit 2024-09-10T18:39:06.576170+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:07.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:07 smithi100 bash[31607]: audit 2024-09-10T18:39:06.577276+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]: dispatch 2024-09-10T18:39:07.996 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: Running main() from gmock_main.cc 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] Global test environment set-up. 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolList (11926 ms) 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3766 ms) 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (5585 ms) 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-10T18:39:07.997 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3689 ms) 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3917 ms) 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5074 ms) 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (6025 ms) 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5079 ms) 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8566 ms) 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] 9 tests from LibRadosPools (53627 ms total) 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: 2024-09-10T18:39:07.998 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] Global test environment tear-down 2024-09-10T18:39:07.999 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (53627 ms total) 2024-09-10T18:39:07.999 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-10T18:39:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[18597]: cluster 2024-09-10T18:39:07.321615+0000 mgr.y (mgr.24512) 200 : cluster [DBG] pgmap v167: 388 pgs: 32 unknown, 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 257 KiB/s wr, 2 op/s 2024-09-10T18:39:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[18597]: audit 2024-09-10T18:39:07.554593+0000 mon.a (mon.0) 1231 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[18597]: cluster 2024-09-10T18:39:07.565426+0000 mon.a (mon.0) 1232 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:39:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[18597]: audit 2024-09-10T18:39:07.992435+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]': finished 2024-09-10T18:39:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[18597]: cluster 2024-09-10T18:39:07.992820+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:39:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[18597]: audit 2024-09-10T18:39:07.996017+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.100:0/1547381286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[18597]: audit 2024-09-10T18:39:07.998580+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[31607]: cluster 2024-09-10T18:39:07.321615+0000 mgr.y (mgr.24512) 200 : cluster [DBG] pgmap v167: 388 pgs: 32 unknown, 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 257 KiB/s wr, 2 op/s 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[31607]: audit 2024-09-10T18:39:07.554593+0000 mon.a (mon.0) 1231 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[31607]: cluster 2024-09-10T18:39:07.565426+0000 mon.a (mon.0) 1232 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[31607]: audit 2024-09-10T18:39:07.992435+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]': finished 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[31607]: cluster 2024-09-10T18:39:07.992820+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[31607]: audit 2024-09-10T18:39:07.996017+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.100:0/1547381286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:08 smithi100 bash[31607]: audit 2024-09-10T18:39:07.998580+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:08.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:08 smithi151 bash[22565]: cluster 2024-09-10T18:39:07.321615+0000 mgr.y (mgr.24512) 200 : cluster [DBG] pgmap v167: 388 pgs: 32 unknown, 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 257 KiB/s wr, 2 op/s 2024-09-10T18:39:08.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:08 smithi151 bash[22565]: audit 2024-09-10T18:39:07.554593+0000 mon.a (mon.0) 1231 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:08.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:08 smithi151 bash[22565]: cluster 2024-09-10T18:39:07.565426+0000 mon.a (mon.0) 1232 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:39:08.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:08 smithi151 bash[22565]: audit 2024-09-10T18:39:07.992435+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-11", "mode": "writeback"}]': finished 2024-09-10T18:39:08.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:08 smithi151 bash[22565]: cluster 2024-09-10T18:39:07.992820+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:39:08.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:08 smithi151 bash[22565]: audit 2024-09-10T18:39:07.996017+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.100:0/1547381286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:08.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:08 smithi151 bash[22565]: audit 2024-09-10T18:39:07.998580+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: audit 2024-09-10T18:39:08.556407+0000 mon.a (mon.0) 1236 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: audit 2024-09-10T18:39:08.578931+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: audit 2024-09-10T18:39:08.579650+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: cluster 2024-09-10T18:39:08.580524+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:39:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: audit 2024-09-10T18:39:08.584876+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? 172.21.15.100:0/3541440877' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: audit 2024-09-10T18:39:08.615602+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: audit 2024-09-10T18:39:08.631295+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[18597]: audit 2024-09-10T18:39:09.321701+0000 mon.c (mon.2) 200 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: audit 2024-09-10T18:39:08.556407+0000 mon.a (mon.0) 1236 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: audit 2024-09-10T18:39:08.578931+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: audit 2024-09-10T18:39:08.579650+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: cluster 2024-09-10T18:39:08.580524+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: audit 2024-09-10T18:39:08.584876+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? 172.21.15.100:0/3541440877' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: audit 2024-09-10T18:39:08.615602+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: audit 2024-09-10T18:39:08.631295+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:09 smithi100 bash[31607]: audit 2024-09-10T18:39:09.321701+0000 mon.c (mon.2) 200 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:09.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: audit 2024-09-10T18:39:08.556407+0000 mon.a (mon.0) 1236 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:09.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: audit 2024-09-10T18:39:08.578931+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-72795-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:09.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: audit 2024-09-10T18:39:08.579650+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-72593-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:09.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: cluster 2024-09-10T18:39:08.580524+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:39:09.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: audit 2024-09-10T18:39:08.584876+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? 172.21.15.100:0/3541440877' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:09.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: audit 2024-09-10T18:39:08.615602+0000 mon.b (mon.1) 82 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:09.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: audit 2024-09-10T18:39:08.631295+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:09.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:09 smithi151 bash[22565]: audit 2024-09-10T18:39:09.321701+0000 mon.c (mon.2) 200 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:11.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: cluster 2024-09-10T18:39:09.322711+0000 mgr.y (mgr.24512) 201 : cluster [DBG] pgmap v170: 396 pgs: 72 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:11.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:09.558182+0000 mon.a (mon.0) 1242 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:11.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:09.570867+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:11.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:09.583352+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? 172.21.15.100:0/3541440877' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:11.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:09.584454+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: cluster 2024-09-10T18:39:09.584556+0000 mon.a (mon.0) 1245 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:09.589101+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: cluster 2024-09-10T18:39:10.267412+0000 mon.a (mon.0) 1247 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: cluster 2024-09-10T18:39:10.269764+0000 mon.a (mon.0) 1248 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:10.286701+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]': finished 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: cluster 2024-09-10T18:39:10.286885+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:10.289898+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.100:0/1758956846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[18597]: audit 2024-09-10T18:39:10.290712+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: cluster 2024-09-10T18:39:09.322711+0000 mgr.y (mgr.24512) 201 : cluster [DBG] pgmap v170: 396 pgs: 72 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:09.558182+0000 mon.a (mon.0) 1242 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:09.570867+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:11.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:09.583352+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? 172.21.15.100:0/3541440877' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:09.584454+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: cluster 2024-09-10T18:39:09.584556+0000 mon.a (mon.0) 1245 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:09.589101+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: cluster 2024-09-10T18:39:10.267412+0000 mon.a (mon.0) 1247 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: cluster 2024-09-10T18:39:10.269764+0000 mon.a (mon.0) 1248 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:10.286701+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]': finished 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: cluster 2024-09-10T18:39:10.286885+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:10.289898+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.100:0/1758956846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:10 smithi100 bash[31607]: audit 2024-09-10T18:39:10.290712+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:11.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: cluster 2024-09-10T18:39:09.322711+0000 mgr.y (mgr.24512) 201 : cluster [DBG] pgmap v170: 396 pgs: 72 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:11.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:09.558182+0000 mon.a (mon.0) 1242 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:11.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:09.570867+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:09.583352+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? 172.21.15.100:0/3541440877' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:09.584454+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: cluster 2024-09-10T18:39:09.584556+0000 mon.a (mon.0) 1245 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:09.589101+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]: dispatch 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: cluster 2024-09-10T18:39:10.267412+0000 mon.a (mon.0) 1247 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: cluster 2024-09-10T18:39:10.269764+0000 mon.a (mon.0) 1248 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:10.286701+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-11"}]': finished 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: cluster 2024-09-10T18:39:10.286885+0000 mon.a (mon.0) 1250 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:10.289898+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.100:0/1758956846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:10 smithi151 bash[22565]: audit 2024-09-10T18:39:10.290712+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:11.755 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:11] "GET /metrics HTTP/1.1" 200 242545 "" "Prometheus/2.43.0" 2024-09-10T18:39:11.756 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18597]: audit 2024-09-10T18:39:10.750286+0000 mon.a (mon.0) 1252 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:11.756 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18597]: cluster 2024-09-10T18:39:10.876602+0000 osd.0 (osd.0) 13 : cluster [DBG] 17.7 deep-scrub starts 2024-09-10T18:39:11.756 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18597]: cluster 2024-09-10T18:39:10.883271+0000 osd.0 (osd.0) 14 : cluster [DBG] 17.7 deep-scrub ok 2024-09-10T18:39:11.756 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18597]: audit 2024-09-10T18:39:11.285556+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:11.756 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18597]: cluster 2024-09-10T18:39:11.285914+0000 mon.a (mon.0) 1254 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:39:11.756 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18597]: audit 2024-09-10T18:39:11.290847+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.100:0/4096394518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:11.756 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[18597]: audit 2024-09-10T18:39:11.724717+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:12.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[31607]: audit 2024-09-10T18:39:10.750286+0000 mon.a (mon.0) 1252 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:12.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[31607]: cluster 2024-09-10T18:39:10.876602+0000 osd.0 (osd.0) 13 : cluster [DBG] 17.7 deep-scrub starts 2024-09-10T18:39:12.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[31607]: cluster 2024-09-10T18:39:10.883271+0000 osd.0 (osd.0) 14 : cluster [DBG] 17.7 deep-scrub ok 2024-09-10T18:39:12.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[31607]: audit 2024-09-10T18:39:11.285556+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:12.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[31607]: cluster 2024-09-10T18:39:11.285914+0000 mon.a (mon.0) 1254 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:39:12.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[31607]: audit 2024-09-10T18:39:11.290847+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.100:0/4096394518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:12.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:11 smithi100 bash[31607]: audit 2024-09-10T18:39:11.724717+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:12.101 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:11 smithi151 bash[22565]: audit 2024-09-10T18:39:10.750286+0000 mon.a (mon.0) 1252 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:12.102 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:11 smithi151 bash[22565]: cluster 2024-09-10T18:39:10.876602+0000 osd.0 (osd.0) 13 : cluster [DBG] 17.7 deep-scrub starts 2024-09-10T18:39:12.102 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:11 smithi151 bash[22565]: cluster 2024-09-10T18:39:10.883271+0000 osd.0 (osd.0) 14 : cluster [DBG] 17.7 deep-scrub ok 2024-09-10T18:39:12.102 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:11 smithi151 bash[22565]: audit 2024-09-10T18:39:11.285556+0000 mon.a (mon.0) 1253 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-72593-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:12.102 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:11 smithi151 bash[22565]: cluster 2024-09-10T18:39:11.285914+0000 mon.a (mon.0) 1254 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:39:12.102 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:11 smithi151 bash[22565]: audit 2024-09-10T18:39:11.290847+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.100:0/4096394518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:12.102 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:11 smithi151 bash[22565]: audit 2024-09-10T18:39:11.724717+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:12.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:39:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:39:13.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[18597]: cluster 2024-09-10T18:39:11.323825+0000 mgr.y (mgr.24512) 202 : cluster [DBG] pgmap v174: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-09-10T18:39:13.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[18597]: audit 2024-09-10T18:39:11.751991+0000 mon.a (mon.0) 1256 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:13.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[18597]: audit 2024-09-10T18:39:12.290131+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:13.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[18597]: audit 2024-09-10T18:39:12.292254+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:13.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[18597]: cluster 2024-09-10T18:39:12.292483+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:39:13.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[18597]: audit 2024-09-10T18:39:12.309153+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[31607]: cluster 2024-09-10T18:39:11.323825+0000 mgr.y (mgr.24512) 202 : cluster [DBG] pgmap v174: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-09-10T18:39:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[31607]: audit 2024-09-10T18:39:11.751991+0000 mon.a (mon.0) 1256 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[31607]: audit 2024-09-10T18:39:12.290131+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[31607]: audit 2024-09-10T18:39:12.292254+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[31607]: cluster 2024-09-10T18:39:12.292483+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:39:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:12 smithi100 bash[31607]: audit 2024-09-10T18:39:12.309153+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:13.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:12 smithi151 bash[22565]: cluster 2024-09-10T18:39:11.323825+0000 mgr.y (mgr.24512) 202 : cluster [DBG] pgmap v174: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-09-10T18:39:13.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:12 smithi151 bash[22565]: audit 2024-09-10T18:39:11.751991+0000 mon.a (mon.0) 1256 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:13.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:12 smithi151 bash[22565]: audit 2024-09-10T18:39:12.290131+0000 mon.b (mon.1) 84 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:13.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:12 smithi151 bash[22565]: audit 2024-09-10T18:39:12.292254+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-72601-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:13.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:12 smithi151 bash[22565]: cluster 2024-09-10T18:39:12.292483+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:39:13.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:12 smithi151 bash[22565]: audit 2024-09-10T18:39:12.309153+0000 mon.a (mon.0) 1259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[18597]: audit 2024-09-10T18:39:12.101569+0000 mgr.y (mgr.24512) 203 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:14.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[18597]: audit 2024-09-10T18:39:12.754701+0000 mon.a (mon.0) 1260 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:14.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[18597]: audit 2024-09-10T18:39:13.300156+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:14.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[18597]: cluster 2024-09-10T18:39:13.300768+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:39:14.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[18597]: audit 2024-09-10T18:39:13.309485+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.100:0/1078028910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[18597]: audit 2024-09-10T18:39:13.310396+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[31607]: audit 2024-09-10T18:39:12.101569+0000 mgr.y (mgr.24512) 203 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[31607]: audit 2024-09-10T18:39:12.754701+0000 mon.a (mon.0) 1260 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[31607]: audit 2024-09-10T18:39:13.300156+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[31607]: cluster 2024-09-10T18:39:13.300768+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:39:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[31607]: audit 2024-09-10T18:39:13.309485+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.100:0/1078028910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:13 smithi100 bash[31607]: audit 2024-09-10T18:39:13.310396+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:13 smithi151 bash[22565]: audit 2024-09-10T18:39:12.101569+0000 mgr.y (mgr.24512) 203 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:14.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:13 smithi151 bash[22565]: audit 2024-09-10T18:39:12.754701+0000 mon.a (mon.0) 1260 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:14.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:13 smithi151 bash[22565]: audit 2024-09-10T18:39:13.300156+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:14.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:13 smithi151 bash[22565]: cluster 2024-09-10T18:39:13.300768+0000 mon.a (mon.0) 1262 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:39:14.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:13 smithi151 bash[22565]: audit 2024-09-10T18:39:13.309485+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.100:0/1078028910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:13 smithi151 bash[22565]: audit 2024-09-10T18:39:13.310396+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:39:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:14.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:14 smithi151 bash[22565]: cluster 2024-09-10T18:39:13.324672+0000 mgr.y (mgr.24512) 204 : cluster [DBG] pgmap v177: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-09-10T18:39:14.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:14 smithi151 bash[22565]: audit 2024-09-10T18:39:13.770306+0000 mon.a (mon.0) 1264 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:14.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:14 smithi151 bash[22565]: audit 2024-09-10T18:39:14.300060+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.100:0/987913630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:14 smithi151 bash[22565]: audit 2024-09-10T18:39:14.309783+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:14 smithi151 bash[22565]: cluster 2024-09-10T18:39:14.310140+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:39:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:14 smithi151 bash[22565]: audit 2024-09-10T18:39:14.316066+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:15.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[18597]: cluster 2024-09-10T18:39:13.324672+0000 mgr.y (mgr.24512) 204 : cluster [DBG] pgmap v177: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-09-10T18:39:15.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[18597]: audit 2024-09-10T18:39:13.770306+0000 mon.a (mon.0) 1264 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:15.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[18597]: audit 2024-09-10T18:39:14.300060+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.100:0/987913630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:15.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[18597]: audit 2024-09-10T18:39:14.309783+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:15.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[18597]: cluster 2024-09-10T18:39:14.310140+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:39:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[18597]: audit 2024-09-10T18:39:14.316066+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[31607]: cluster 2024-09-10T18:39:13.324672+0000 mgr.y (mgr.24512) 204 : cluster [DBG] pgmap v177: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.9 KiB/s wr, 3 op/s 2024-09-10T18:39:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[31607]: audit 2024-09-10T18:39:13.770306+0000 mon.a (mon.0) 1264 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[31607]: audit 2024-09-10T18:39:14.300060+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.100:0/987913630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[31607]: audit 2024-09-10T18:39:14.309783+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi100-72593-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[31607]: cluster 2024-09-10T18:39:14.310140+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:39:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:14 smithi100 bash[31607]: audit 2024-09-10T18:39:14.316066+0000 mon.a (mon.0) 1267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:16.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:15 smithi100 bash[18597]: audit 2024-09-10T18:39:14.772238+0000 mon.a (mon.0) 1268 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:16.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:15 smithi100 bash[18597]: audit 2024-09-10T18:39:15.283828+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:16.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:15 smithi100 bash[18597]: cluster 2024-09-10T18:39:15.285267+0000 mon.a (mon.0) 1270 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:39:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:15 smithi100 bash[31607]: audit 2024-09-10T18:39:14.772238+0000 mon.a (mon.0) 1268 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:15 smithi100 bash[31607]: audit 2024-09-10T18:39:15.283828+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:15 smithi100 bash[31607]: cluster 2024-09-10T18:39:15.285267+0000 mon.a (mon.0) 1270 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:39:16.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:15 smithi151 bash[22565]: audit 2024-09-10T18:39:14.772238+0000 mon.a (mon.0) 1268 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:16.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:15 smithi151 bash[22565]: audit 2024-09-10T18:39:15.283828+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-72601-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:16.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:15 smithi151 bash[22565]: cluster 2024-09-10T18:39:15.285267+0000 mon.a (mon.0) 1270 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:39:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[18597]: cluster 2024-09-10T18:39:15.325402+0000 mgr.y (mgr.24512) 205 : cluster [DBG] pgmap v180: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 56 KiB/s wr, 52 op/s 2024-09-10T18:39:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[18597]: audit 2024-09-10T18:39:15.774139+0000 mon.a (mon.0) 1271 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[18597]: cluster 2024-09-10T18:39:16.276090+0000 mon.a (mon.0) 1272 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[18597]: cluster 2024-09-10T18:39:16.293468+0000 mon.a (mon.0) 1273 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:39:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[18597]: audit 2024-09-10T18:39:16.299272+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.100:0/4239917258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[18597]: audit 2024-09-10T18:39:16.299852+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[31607]: cluster 2024-09-10T18:39:15.325402+0000 mgr.y (mgr.24512) 205 : cluster [DBG] pgmap v180: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 56 KiB/s wr, 52 op/s 2024-09-10T18:39:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[31607]: audit 2024-09-10T18:39:15.774139+0000 mon.a (mon.0) 1271 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[31607]: cluster 2024-09-10T18:39:16.276090+0000 mon.a (mon.0) 1272 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[31607]: cluster 2024-09-10T18:39:16.293468+0000 mon.a (mon.0) 1273 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:39:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[31607]: audit 2024-09-10T18:39:16.299272+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.100:0/4239917258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:16 smithi100 bash[31607]: audit 2024-09-10T18:39:16.299852+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:17.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:16 smithi151 bash[22565]: cluster 2024-09-10T18:39:15.325402+0000 mgr.y (mgr.24512) 205 : cluster [DBG] pgmap v180: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 56 KiB/s wr, 52 op/s 2024-09-10T18:39:17.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:16 smithi151 bash[22565]: audit 2024-09-10T18:39:15.774139+0000 mon.a (mon.0) 1271 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:17.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:16 smithi151 bash[22565]: cluster 2024-09-10T18:39:16.276090+0000 mon.a (mon.0) 1272 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:17.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:16 smithi151 bash[22565]: cluster 2024-09-10T18:39:16.293468+0000 mon.a (mon.0) 1273 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:39:17.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:16 smithi151 bash[22565]: audit 2024-09-10T18:39:16.299272+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.100:0/4239917258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:17.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:16 smithi151 bash[22565]: audit 2024-09-10T18:39:16.299852+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: audit 2024-09-10T18:39:16.776455+0000 mon.a (mon.0) 1275 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: audit 2024-09-10T18:39:17.298741+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: cluster 2024-09-10T18:39:17.300215+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:39:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: audit 2024-09-10T18:39:17.304823+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: audit 2024-09-10T18:39:17.306811+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.100:0/1120560187' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: audit 2024-09-10T18:39:17.308029+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: audit 2024-09-10T18:39:17.568399+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[18597]: audit 2024-09-10T18:39:17.584476+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: audit 2024-09-10T18:39:16.776455+0000 mon.a (mon.0) 1275 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: audit 2024-09-10T18:39:17.298741+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: cluster 2024-09-10T18:39:17.300215+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: audit 2024-09-10T18:39:17.304823+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: audit 2024-09-10T18:39:17.306811+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.100:0/1120560187' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: audit 2024-09-10T18:39:17.308029+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:18.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: audit 2024-09-10T18:39:17.568399+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:18.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:17 smithi100 bash[31607]: audit 2024-09-10T18:39:17.584476+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:18.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: audit 2024-09-10T18:39:16.776455+0000 mon.a (mon.0) 1275 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:18.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: audit 2024-09-10T18:39:17.298741+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-72593-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:18.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: cluster 2024-09-10T18:39:17.300215+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:39:18.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: audit 2024-09-10T18:39:17.304823+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:18.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: audit 2024-09-10T18:39:17.306811+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.100:0/1120560187' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:18.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: audit 2024-09-10T18:39:17.308029+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:18.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: audit 2024-09-10T18:39:17.568399+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:18.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:17 smithi151 bash[22565]: audit 2024-09-10T18:39:17.584476+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:39:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: cluster 2024-09-10T18:39:17.326110+0000 mgr.y (mgr.24512) 206 : cluster [DBG] pgmap v183: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 48 KiB/s wr, 50 op/s 2024-09-10T18:39:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: audit 2024-09-10T18:39:17.778478+0000 mon.a (mon.0) 1281 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: audit 2024-09-10T18:39:18.294761+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: audit 2024-09-10T18:39:18.306528+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: audit 2024-09-10T18:39:18.306804+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: audit 2024-09-10T18:39:18.307123+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: cluster 2024-09-10T18:39:18.307272+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: audit 2024-09-10T18:39:18.309162+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[18597]: audit 2024-09-10T18:39:18.312583+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: cluster 2024-09-10T18:39:17.326110+0000 mgr.y (mgr.24512) 206 : cluster [DBG] pgmap v183: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 48 KiB/s wr, 50 op/s 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: audit 2024-09-10T18:39:17.778478+0000 mon.a (mon.0) 1281 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: audit 2024-09-10T18:39:18.294761+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: audit 2024-09-10T18:39:18.306528+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: audit 2024-09-10T18:39:18.306804+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: audit 2024-09-10T18:39:18.307123+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:39:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: cluster 2024-09-10T18:39:18.307272+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:39:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: audit 2024-09-10T18:39:18.309162+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:18 smithi100 bash[31607]: audit 2024-09-10T18:39:18.312583+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:19.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: cluster 2024-09-10T18:39:17.326110+0000 mgr.y (mgr.24512) 206 : cluster [DBG] pgmap v183: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 48 KiB/s wr, 50 op/s 2024-09-10T18:39:19.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: audit 2024-09-10T18:39:17.778478+0000 mon.a (mon.0) 1281 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:19.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: audit 2024-09-10T18:39:18.294761+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:19.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: audit 2024-09-10T18:39:18.306528+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:19.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: audit 2024-09-10T18:39:18.306804+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-72601-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:19.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: audit 2024-09-10T18:39:18.307123+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:39:19.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: cluster 2024-09-10T18:39:18.307272+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:39:19.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: audit 2024-09-10T18:39:18.309162+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]: dispatch 2024-09-10T18:39:19.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:18 smithi151 bash[22565]: audit 2024-09-10T18:39:18.312583+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:19.321 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-10T18:39:19.321 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-09-10T18:39:19.321 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-10T18:39:19.321 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-10T18:39:19.321 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-10T18:39:19.321 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2357 ms) 2024-09-10T18:39:19.321 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3190 ms) 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2766 ms) 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2377 ms) 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (10690 ms total) 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4233 ms) 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (3874 ms) 2024-09-10T18:39:19.322 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (8107 ms total) 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2965 ms) 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2295 ms) 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1910 ms) 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2255 ms) 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9425 ms total) 2024-09-10T18:39:19.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3741 ms) 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3991 ms) 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (7732 ms total) 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-09-10T18:39:19.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-10T18:39:19.325 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (64940 ms total) 2024-09-10T18:39:19.325 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-09-10T18:39:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: audit 2024-09-10T18:39:18.781407+0000 mon.a (mon.0) 1288 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: audit 2024-09-10T18:39:19.305040+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]: dispatch 2024-09-10T18:39:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: audit 2024-09-10T18:39:19.313307+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: audit 2024-09-10T18:39:19.313554+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]': finished 2024-09-10T18:39:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: cluster 2024-09-10T18:39:19.314183+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:39:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: audit 2024-09-10T18:39:19.317411+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? 172.21.15.100:0/1687220441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-72593-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: audit 2024-09-10T18:39:19.322516+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]: dispatch 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[18597]: audit 2024-09-10T18:39:19.783040+0000 mon.a (mon.0) 1294 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: audit 2024-09-10T18:39:18.781407+0000 mon.a (mon.0) 1288 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: audit 2024-09-10T18:39:19.305040+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]: dispatch 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: audit 2024-09-10T18:39:19.313307+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: audit 2024-09-10T18:39:19.313554+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]': finished 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: cluster 2024-09-10T18:39:19.314183+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: audit 2024-09-10T18:39:19.317411+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? 172.21.15.100:0/1687220441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-72593-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: audit 2024-09-10T18:39:19.322516+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]: dispatch 2024-09-10T18:39:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:19 smithi100 bash[31607]: audit 2024-09-10T18:39:19.783040+0000 mon.a (mon.0) 1294 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:20.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: audit 2024-09-10T18:39:18.781407+0000 mon.a (mon.0) 1288 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:20.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: audit 2024-09-10T18:39:19.305040+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]: dispatch 2024-09-10T18:39:20.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: audit 2024-09-10T18:39:19.313307+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? 172.21.15.100:0/2809150383' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-72795-14"}]': finished 2024-09-10T18:39:20.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: audit 2024-09-10T18:39:19.313554+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-13"}]': finished 2024-09-10T18:39:20.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: cluster 2024-09-10T18:39:19.314183+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:39:20.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: audit 2024-09-10T18:39:19.317411+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? 172.21.15.100:0/1687220441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-72593-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:20.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: audit 2024-09-10T18:39:19.322516+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]: dispatch 2024-09-10T18:39:20.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:19 smithi151 bash[22565]: audit 2024-09-10T18:39:19.783040+0000 mon.a (mon.0) 1294 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: cluster 2024-09-10T18:39:19.326768+0000 mgr.y (mgr.24512) 207 : cluster [DBG] pgmap v186: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: cluster 2024-09-10T18:39:20.277256+0000 mon.a (mon.0) 1295 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:39:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: audit 2024-09-10T18:39:20.293323+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.100:0/1687220441' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-72593-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: audit 2024-09-10T18:39:20.293586+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]': finished 2024-09-10T18:39:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: cluster 2024-09-10T18:39:20.293898+0000 mon.a (mon.0) 1298 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:39:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: audit 2024-09-10T18:39:20.295519+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.100:0/728543267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: audit 2024-09-10T18:39:20.296999+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[18597]: audit 2024-09-10T18:39:20.784620+0000 mon.a (mon.0) 1300 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: cluster 2024-09-10T18:39:19.326768+0000 mgr.y (mgr.24512) 207 : cluster [DBG] pgmap v186: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: cluster 2024-09-10T18:39:20.277256+0000 mon.a (mon.0) 1295 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: audit 2024-09-10T18:39:20.293323+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.100:0/1687220441' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-72593-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: audit 2024-09-10T18:39:20.293586+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]': finished 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: cluster 2024-09-10T18:39:20.293898+0000 mon.a (mon.0) 1298 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: audit 2024-09-10T18:39:20.295519+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.100:0/728543267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: audit 2024-09-10T18:39:20.296999+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:20 smithi100 bash[31607]: audit 2024-09-10T18:39:20.784620+0000 mon.a (mon.0) 1300 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:21.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: cluster 2024-09-10T18:39:19.326768+0000 mgr.y (mgr.24512) 207 : cluster [DBG] pgmap v186: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:21.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: cluster 2024-09-10T18:39:20.277256+0000 mon.a (mon.0) 1295 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:39:21.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: audit 2024-09-10T18:39:20.293323+0000 mon.a (mon.0) 1296 : audit [INF] from='client.? 172.21.15.100:0/1687220441' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-72593-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:21.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: audit 2024-09-10T18:39:20.293586+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-13", "mode": "writeback"}]': finished 2024-09-10T18:39:21.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: cluster 2024-09-10T18:39:20.293898+0000 mon.a (mon.0) 1298 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:39:21.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: audit 2024-09-10T18:39:20.295519+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.100:0/728543267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:21.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: audit 2024-09-10T18:39:20.296999+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:21.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:20 smithi151 bash[22565]: audit 2024-09-10T18:39:20.784620+0000 mon.a (mon.0) 1300 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:39:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:21] "GET /metrics HTTP/1.1" 200 242545 "" "Prometheus/2.43.0" 2024-09-10T18:39:22.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:39:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.293611+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: cluster 2024-09-10T18:39:21.294150+0000 mon.a (mon.0) 1302 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:39:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: cluster 2024-09-10T18:39:21.327443+0000 mgr.y (mgr.24512) 208 : cluster [DBG] pgmap v189: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-09-10T18:39:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.474366+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-09-10T18:39:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.475795+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-09-10T18:39:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.477128+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-09-10T18:39:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.478374+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.479532+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.480695+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.481791+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.482838+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.484007+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.485128+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.490174+0000 mgr.y (mgr.24512) 209 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.491492+0000 mgr.y (mgr.24512) 210 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.492824+0000 mgr.y (mgr.24512) 211 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.494039+0000 mgr.y (mgr.24512) 212 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-09-10T18:39:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.495196+0000 mgr.y (mgr.24512) 213 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.496357+0000 mgr.y (mgr.24512) 214 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.497442+0000 mgr.y (mgr.24512) 215 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.498482+0000 mgr.y (mgr.24512) 216 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.499660+0000 mgr.y (mgr.24512) 217 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.500759+0000 mgr.y (mgr.24512) 218 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: cluster 2024-09-10T18:39:21.738390+0000 osd.0 (osd.0) 15 : cluster [DBG] 96.4 scrub starts 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: cluster 2024-09-10T18:39:21.743481+0000 osd.0 (osd.0) 16 : cluster [DBG] 96.4 scrub ok 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: cluster 2024-09-10T18:39:21.763164+0000 osd.5 (osd.5) 3 : cluster [DBG] 96.7 scrub starts 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: cluster 2024-09-10T18:39:21.769938+0000 osd.5 (osd.5) 4 : cluster [DBG] 96.7 scrub ok 2024-09-10T18:39:22.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:22 smithi151 bash[22565]: audit 2024-09-10T18:39:21.786049+0000 mon.a (mon.0) 1303 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.293611+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: cluster 2024-09-10T18:39:21.294150+0000 mon.a (mon.0) 1302 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: cluster 2024-09-10T18:39:21.327443+0000 mgr.y (mgr.24512) 208 : cluster [DBG] pgmap v189: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.474366+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.475795+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.477128+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.478374+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.479532+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.480695+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.481791+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.482838+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.484007+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.485128+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-09-10T18:39:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.490174+0000 mgr.y (mgr.24512) 209 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-09-10T18:39:22.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.491492+0000 mgr.y (mgr.24512) 210 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-09-10T18:39:22.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.492824+0000 mgr.y (mgr.24512) 211 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-09-10T18:39:22.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.494039+0000 mgr.y (mgr.24512) 212 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-09-10T18:39:22.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.495196+0000 mgr.y (mgr.24512) 213 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-09-10T18:39:22.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.293611+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-72601-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:22.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: cluster 2024-09-10T18:39:21.294150+0000 mon.a (mon.0) 1302 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:39:22.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: cluster 2024-09-10T18:39:21.327443+0000 mgr.y (mgr.24512) 208 : cluster [DBG] pgmap v189: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.474366+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.475795+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.477128+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.478374+0000 mon.b (mon.1) 92 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.479532+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.480695+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.481791+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.482838+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.484007+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.485128+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.490174+0000 mgr.y (mgr.24512) 209 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.491492+0000 mgr.y (mgr.24512) 210 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-09-10T18:39:22.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.492824+0000 mgr.y (mgr.24512) 211 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.494039+0000 mgr.y (mgr.24512) 212 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.495196+0000 mgr.y (mgr.24512) 213 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.496357+0000 mgr.y (mgr.24512) 214 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.497442+0000 mgr.y (mgr.24512) 215 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.498482+0000 mgr.y (mgr.24512) 216 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.499660+0000 mgr.y (mgr.24512) 217 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.500759+0000 mgr.y (mgr.24512) 218 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: cluster 2024-09-10T18:39:21.738390+0000 osd.0 (osd.0) 15 : cluster [DBG] 96.4 scrub starts 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: cluster 2024-09-10T18:39:21.743481+0000 osd.0 (osd.0) 16 : cluster [DBG] 96.4 scrub ok 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: cluster 2024-09-10T18:39:21.763164+0000 osd.5 (osd.5) 3 : cluster [DBG] 96.7 scrub starts 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: cluster 2024-09-10T18:39:21.769938+0000 osd.5 (osd.5) 4 : cluster [DBG] 96.7 scrub ok 2024-09-10T18:39:22.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[31607]: audit 2024-09-10T18:39:21.786049+0000 mon.a (mon.0) 1303 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.496357+0000 mgr.y (mgr.24512) 214 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.497442+0000 mgr.y (mgr.24512) 215 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.498482+0000 mgr.y (mgr.24512) 216 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.499660+0000 mgr.y (mgr.24512) 217 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.500759+0000 mgr.y (mgr.24512) 218 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: cluster 2024-09-10T18:39:21.738390+0000 osd.0 (osd.0) 15 : cluster [DBG] 96.4 scrub starts 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: cluster 2024-09-10T18:39:21.743481+0000 osd.0 (osd.0) 16 : cluster [DBG] 96.4 scrub ok 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: cluster 2024-09-10T18:39:21.763164+0000 osd.5 (osd.5) 3 : cluster [DBG] 96.7 scrub starts 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: cluster 2024-09-10T18:39:21.769938+0000 osd.5 (osd.5) 4 : cluster [DBG] 96.7 scrub ok 2024-09-10T18:39:22.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:22 smithi100 bash[18597]: audit 2024-09-10T18:39:21.786049+0000 mon.a (mon.0) 1303 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:23.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: audit 2024-09-10T18:39:22.113020+0000 mgr.y (mgr.24512) 219 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: cluster 2024-09-10T18:39:22.284738+0000 mon.a (mon.0) 1304 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: cluster 2024-09-10T18:39:22.292618+0000 osd.2 (osd.2) 3 : cluster [DBG] 96.0 scrub starts 2024-09-10T18:39:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: cluster 2024-09-10T18:39:22.298596+0000 osd.2 (osd.2) 4 : cluster [DBG] 96.0 scrub ok 2024-09-10T18:39:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: cluster 2024-09-10T18:39:22.310110+0000 mon.a (mon.0) 1305 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:39:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: audit 2024-09-10T18:39:22.312224+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.100:0/1029854552' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: audit 2024-09-10T18:39:22.313811+0000 mon.a (mon.0) 1306 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:23 smithi151 bash[22565]: audit 2024-09-10T18:39:22.787764+0000 mon.a (mon.0) 1307 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: audit 2024-09-10T18:39:22.113020+0000 mgr.y (mgr.24512) 219 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: cluster 2024-09-10T18:39:22.284738+0000 mon.a (mon.0) 1304 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: cluster 2024-09-10T18:39:22.292618+0000 osd.2 (osd.2) 3 : cluster [DBG] 96.0 scrub starts 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: cluster 2024-09-10T18:39:22.298596+0000 osd.2 (osd.2) 4 : cluster [DBG] 96.0 scrub ok 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: cluster 2024-09-10T18:39:22.310110+0000 mon.a (mon.0) 1305 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: audit 2024-09-10T18:39:22.312224+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.100:0/1029854552' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: audit 2024-09-10T18:39:22.313811+0000 mon.a (mon.0) 1306 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[18597]: audit 2024-09-10T18:39:22.787764+0000 mon.a (mon.0) 1307 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: audit 2024-09-10T18:39:22.113020+0000 mgr.y (mgr.24512) 219 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: cluster 2024-09-10T18:39:22.284738+0000 mon.a (mon.0) 1304 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: cluster 2024-09-10T18:39:22.292618+0000 osd.2 (osd.2) 3 : cluster [DBG] 96.0 scrub starts 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: cluster 2024-09-10T18:39:22.298596+0000 osd.2 (osd.2) 4 : cluster [DBG] 96.0 scrub ok 2024-09-10T18:39:23.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: cluster 2024-09-10T18:39:22.310110+0000 mon.a (mon.0) 1305 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:39:23.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: audit 2024-09-10T18:39:22.312224+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.100:0/1029854552' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:23.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: audit 2024-09-10T18:39:22.313811+0000 mon.a (mon.0) 1306 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:23.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:23 smithi100 bash[31607]: audit 2024-09-10T18:39:22.787764+0000 mon.a (mon.0) 1307 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:24.677 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:24.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: audit 2024-09-10T18:39:23.300044+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.100:0/3368312319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: audit 2024-09-10T18:39:23.312283+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: cluster 2024-09-10T18:39:23.312666+0000 mon.a (mon.0) 1309 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:39:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: audit 2024-09-10T18:39:23.325925+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: cluster 2024-09-10T18:39:23.328144+0000 mgr.y (mgr.24512) 220 : cluster [DBG] pgmap v192: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-09-10T18:39:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: cluster 2024-09-10T18:39:23.399786+0000 osd.3 (osd.3) 7 : cluster [DBG] 96.6 scrub starts 2024-09-10T18:39:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: cluster 2024-09-10T18:39:23.405138+0000 osd.3 (osd.3) 8 : cluster [DBG] 96.6 scrub ok 2024-09-10T18:39:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:24 smithi151 bash[22565]: audit 2024-09-10T18:39:23.789421+0000 mon.a (mon.0) 1311 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: audit 2024-09-10T18:39:23.300044+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.100:0/3368312319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: audit 2024-09-10T18:39:23.312283+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: cluster 2024-09-10T18:39:23.312666+0000 mon.a (mon.0) 1309 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:39:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: audit 2024-09-10T18:39:23.325925+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: cluster 2024-09-10T18:39:23.328144+0000 mgr.y (mgr.24512) 220 : cluster [DBG] pgmap v192: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-09-10T18:39:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: cluster 2024-09-10T18:39:23.399786+0000 osd.3 (osd.3) 7 : cluster [DBG] 96.6 scrub starts 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: cluster 2024-09-10T18:39:23.405138+0000 osd.3 (osd.3) 8 : cluster [DBG] 96.6 scrub ok 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[18597]: audit 2024-09-10T18:39:23.789421+0000 mon.a (mon.0) 1311 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: audit 2024-09-10T18:39:23.300044+0000 mon.b (mon.1) 99 : audit [INF] from='client.? 172.21.15.100:0/3368312319' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: audit 2024-09-10T18:39:23.312283+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-72593-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: cluster 2024-09-10T18:39:23.312666+0000 mon.a (mon.0) 1309 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: audit 2024-09-10T18:39:23.325925+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: cluster 2024-09-10T18:39:23.328144+0000 mgr.y (mgr.24512) 220 : cluster [DBG] pgmap v192: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: cluster 2024-09-10T18:39:23.399786+0000 osd.3 (osd.3) 7 : cluster [DBG] 96.6 scrub starts 2024-09-10T18:39:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: cluster 2024-09-10T18:39:23.405138+0000 osd.3 (osd.3) 8 : cluster [DBG] 96.6 scrub ok 2024-09-10T18:39:24.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:24 smithi100 bash[31607]: audit 2024-09-10T18:39:23.789421+0000 mon.a (mon.0) 1311 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:25.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: cluster 2024-09-10T18:39:23.519664+0000 osd.7 (osd.7) 11 : cluster [DBG] 96.5 scrub starts 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: cluster 2024-09-10T18:39:23.525410+0000 osd.7 (osd.7) 12 : cluster [DBG] 96.5 scrub ok 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: audit 2024-09-10T18:39:24.314614+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: cluster 2024-09-10T18:39:24.316249+0000 mon.a (mon.0) 1313 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: audit 2024-09-10T18:39:24.322807+0000 mon.c (mon.2) 208 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: audit 2024-09-10T18:39:25.219597+0000 mon.a (mon.0) 1314 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: audit 2024-09-10T18:39:25.323991+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.100:0/653439856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: cluster 2024-09-10T18:39:25.336952+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: audit 2024-09-10T18:39:25.339603+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? 172.21.15.100:0/1762975413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72806-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:25 smithi151 bash[22565]: audit 2024-09-10T18:39:25.340644+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: cluster 2024-09-10T18:39:23.519664+0000 osd.7 (osd.7) 11 : cluster [DBG] 96.5 scrub starts 2024-09-10T18:39:25.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: cluster 2024-09-10T18:39:23.525410+0000 osd.7 (osd.7) 12 : cluster [DBG] 96.5 scrub ok 2024-09-10T18:39:25.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: audit 2024-09-10T18:39:24.314614+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: cluster 2024-09-10T18:39:24.316249+0000 mon.a (mon.0) 1313 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: audit 2024-09-10T18:39:24.322807+0000 mon.c (mon.2) 208 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: audit 2024-09-10T18:39:25.219597+0000 mon.a (mon.0) 1314 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: audit 2024-09-10T18:39:25.323991+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.100:0/653439856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: cluster 2024-09-10T18:39:25.336952+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: audit 2024-09-10T18:39:25.339603+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? 172.21.15.100:0/1762975413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72806-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[31607]: audit 2024-09-10T18:39:25.340644+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: cluster 2024-09-10T18:39:23.519664+0000 osd.7 (osd.7) 11 : cluster [DBG] 96.5 scrub starts 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: cluster 2024-09-10T18:39:23.525410+0000 osd.7 (osd.7) 12 : cluster [DBG] 96.5 scrub ok 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: audit 2024-09-10T18:39:24.314614+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-72601-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: cluster 2024-09-10T18:39:24.316249+0000 mon.a (mon.0) 1313 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:39:25.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: audit 2024-09-10T18:39:24.322807+0000 mon.c (mon.2) 208 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:25.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: audit 2024-09-10T18:39:25.219597+0000 mon.a (mon.0) 1314 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:25.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: audit 2024-09-10T18:39:25.323991+0000 mon.b (mon.1) 100 : audit [INF] from='client.? 172.21.15.100:0/653439856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: cluster 2024-09-10T18:39:25.336952+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:39:25.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: audit 2024-09-10T18:39:25.339603+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? 172.21.15.100:0/1762975413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72806-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:25.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:25 smithi100 bash[18597]: audit 2024-09-10T18:39:25.340644+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:26.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: cluster 2024-09-10T18:39:24.842249+0000 osd.1 (osd.1) 3 : cluster [DBG] 96.3 scrub starts 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: cluster 2024-09-10T18:39:24.845515+0000 osd.1 (osd.1) 4 : cluster [DBG] 96.3 scrub ok 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: cluster 2024-09-10T18:39:25.307938+0000 osd.2 (osd.2) 5 : cluster [DBG] 96.9 scrub starts 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: cluster 2024-09-10T18:39:25.315037+0000 osd.2 (osd.2) 6 : cluster [DBG] 96.9 scrub ok 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: cluster 2024-09-10T18:39:25.330024+0000 mgr.y (mgr.24512) 221 : cluster [DBG] pgmap v194: 324 pgs: 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 73 op/s 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: audit 2024-09-10T18:39:26.221358+0000 mon.a (mon.0) 1318 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: audit 2024-09-10T18:39:26.349018+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? 172.21.15.100:0/1762975413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72806-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: audit 2024-09-10T18:39:26.349198+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:26 smithi151 bash[22565]: cluster 2024-09-10T18:39:26.349768+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:39:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: cluster 2024-09-10T18:39:24.842249+0000 osd.1 (osd.1) 3 : cluster [DBG] 96.3 scrub starts 2024-09-10T18:39:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: cluster 2024-09-10T18:39:24.845515+0000 osd.1 (osd.1) 4 : cluster [DBG] 96.3 scrub ok 2024-09-10T18:39:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: cluster 2024-09-10T18:39:25.307938+0000 osd.2 (osd.2) 5 : cluster [DBG] 96.9 scrub starts 2024-09-10T18:39:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: cluster 2024-09-10T18:39:25.315037+0000 osd.2 (osd.2) 6 : cluster [DBG] 96.9 scrub ok 2024-09-10T18:39:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: cluster 2024-09-10T18:39:25.330024+0000 mgr.y (mgr.24512) 221 : cluster [DBG] pgmap v194: 324 pgs: 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 73 op/s 2024-09-10T18:39:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: audit 2024-09-10T18:39:26.221358+0000 mon.a (mon.0) 1318 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: audit 2024-09-10T18:39:26.349018+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? 172.21.15.100:0/1762975413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72806-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: audit 2024-09-10T18:39:26.349198+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[18597]: cluster 2024-09-10T18:39:26.349768+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: cluster 2024-09-10T18:39:24.842249+0000 osd.1 (osd.1) 3 : cluster [DBG] 96.3 scrub starts 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: cluster 2024-09-10T18:39:24.845515+0000 osd.1 (osd.1) 4 : cluster [DBG] 96.3 scrub ok 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: cluster 2024-09-10T18:39:25.307938+0000 osd.2 (osd.2) 5 : cluster [DBG] 96.9 scrub starts 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: cluster 2024-09-10T18:39:25.315037+0000 osd.2 (osd.2) 6 : cluster [DBG] 96.9 scrub ok 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: cluster 2024-09-10T18:39:25.330024+0000 mgr.y (mgr.24512) 221 : cluster [DBG] pgmap v194: 324 pgs: 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 73 op/s 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: audit 2024-09-10T18:39:26.221358+0000 mon.a (mon.0) 1318 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: audit 2024-09-10T18:39:26.349018+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? 172.21.15.100:0/1762975413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72806-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: audit 2024-09-10T18:39:26.349198+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-72593-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:26 smithi100 bash[31607]: cluster 2024-09-10T18:39:26.349768+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: cluster 2024-09-10T18:39:25.429888+0000 osd.4 (osd.4) 11 : cluster [DBG] 96.2 scrub starts 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: cluster 2024-09-10T18:39:25.435226+0000 osd.4 (osd.4) 12 : cluster [DBG] 96.2 scrub ok 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: cluster 2024-09-10T18:39:26.325237+0000 osd.6 (osd.6) 11 : cluster [DBG] 96.8 scrub starts 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: cluster 2024-09-10T18:39:26.334169+0000 osd.6 (osd.6) 12 : cluster [DBG] 96.8 scrub ok 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: audit 2024-09-10T18:39:26.355782+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.100:0/85067972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: audit 2024-09-10T18:39:26.364913+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: audit 2024-09-10T18:39:27.223086+0000 mon.a (mon.0) 1323 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:27.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:27 smithi151 bash[22565]: audit 2024-09-10T18:39:27.224437+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:39:27.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: cluster 2024-09-10T18:39:25.429888+0000 osd.4 (osd.4) 11 : cluster [DBG] 96.2 scrub starts 2024-09-10T18:39:27.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: cluster 2024-09-10T18:39:25.435226+0000 osd.4 (osd.4) 12 : cluster [DBG] 96.2 scrub ok 2024-09-10T18:39:27.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: cluster 2024-09-10T18:39:26.325237+0000 osd.6 (osd.6) 11 : cluster [DBG] 96.8 scrub starts 2024-09-10T18:39:27.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: cluster 2024-09-10T18:39:26.334169+0000 osd.6 (osd.6) 12 : cluster [DBG] 96.8 scrub ok 2024-09-10T18:39:27.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: audit 2024-09-10T18:39:26.355782+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.100:0/85067972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:27.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: audit 2024-09-10T18:39:26.364913+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:27.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: audit 2024-09-10T18:39:27.223086+0000 mon.a (mon.0) 1323 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[31607]: audit 2024-09-10T18:39:27.224437+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: cluster 2024-09-10T18:39:25.429888+0000 osd.4 (osd.4) 11 : cluster [DBG] 96.2 scrub starts 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: cluster 2024-09-10T18:39:25.435226+0000 osd.4 (osd.4) 12 : cluster [DBG] 96.2 scrub ok 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: cluster 2024-09-10T18:39:26.325237+0000 osd.6 (osd.6) 11 : cluster [DBG] 96.8 scrub starts 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: cluster 2024-09-10T18:39:26.334169+0000 osd.6 (osd.6) 12 : cluster [DBG] 96.8 scrub ok 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: audit 2024-09-10T18:39:26.355782+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.100:0/85067972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: audit 2024-09-10T18:39:26.364913+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: audit 2024-09-10T18:39:27.223086+0000 mon.a (mon.0) 1323 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:27 smithi100 bash[18597]: audit 2024-09-10T18:39:27.224437+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:39:28.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:28 smithi151 bash[22565]: cluster 2024-09-10T18:39:27.330986+0000 mgr.y (mgr.24512) 222 : cluster [DBG] pgmap v197: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 73 op/s 2024-09-10T18:39:28.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:28 smithi151 bash[22565]: cluster 2024-09-10T18:39:27.360247+0000 mon.a (mon.0) 1325 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:28.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:28 smithi151 bash[22565]: audit 2024-09-10T18:39:27.376648+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:28.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:28 smithi151 bash[22565]: audit 2024-09-10T18:39:27.377185+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:39:28.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:28 smithi151 bash[22565]: cluster 2024-09-10T18:39:27.377294+0000 mon.a (mon.0) 1328 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:39:28.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:28 smithi151 bash[22565]: audit 2024-09-10T18:39:27.378316+0000 mon.a (mon.0) 1329 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[31607]: cluster 2024-09-10T18:39:27.330986+0000 mgr.y (mgr.24512) 222 : cluster [DBG] pgmap v197: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 73 op/s 2024-09-10T18:39:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[31607]: cluster 2024-09-10T18:39:27.360247+0000 mon.a (mon.0) 1325 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[31607]: audit 2024-09-10T18:39:27.376648+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[31607]: audit 2024-09-10T18:39:27.377185+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:39:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[31607]: cluster 2024-09-10T18:39:27.377294+0000 mon.a (mon.0) 1328 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:39:28.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[31607]: audit 2024-09-10T18:39:27.378316+0000 mon.a (mon.0) 1329 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[18597]: cluster 2024-09-10T18:39:27.330986+0000 mgr.y (mgr.24512) 222 : cluster [DBG] pgmap v197: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 55 KiB/s rd, 73 op/s 2024-09-10T18:39:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[18597]: cluster 2024-09-10T18:39:27.360247+0000 mon.a (mon.0) 1325 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[18597]: audit 2024-09-10T18:39:27.376648+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-72601-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[18597]: audit 2024-09-10T18:39:27.377185+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:39:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[18597]: cluster 2024-09-10T18:39:27.377294+0000 mon.a (mon.0) 1328 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:39:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:28 smithi100 bash[18597]: audit 2024-09-10T18:39:27.378316+0000 mon.a (mon.0) 1329 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:29.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:29 smithi151 bash[22565]: cluster 2024-09-10T18:39:27.403069+0000 osd.4 (osd.4) 13 : cluster [DBG] 96.1 scrub starts 2024-09-10T18:39:29.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:29 smithi151 bash[22565]: cluster 2024-09-10T18:39:28.383499+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:39:29.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:29 smithi151 bash[22565]: audit 2024-09-10T18:39:28.384703+0000 mon.a (mon.0) 1331 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:29.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:29 smithi151 bash[22565]: audit 2024-09-10T18:39:28.387018+0000 mon.a (mon.0) 1332 : audit [INF] from='client.? 172.21.15.100:0/2855743240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72593-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[18597]: cluster 2024-09-10T18:39:27.403069+0000 osd.4 (osd.4) 13 : cluster [DBG] 96.1 scrub starts 2024-09-10T18:39:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[18597]: cluster 2024-09-10T18:39:28.383499+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:39:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[18597]: audit 2024-09-10T18:39:28.384703+0000 mon.a (mon.0) 1331 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[18597]: audit 2024-09-10T18:39:28.387018+0000 mon.a (mon.0) 1332 : audit [INF] from='client.? 172.21.15.100:0/2855743240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72593-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[31607]: cluster 2024-09-10T18:39:27.403069+0000 osd.4 (osd.4) 13 : cluster [DBG] 96.1 scrub starts 2024-09-10T18:39:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[31607]: cluster 2024-09-10T18:39:28.383499+0000 mon.a (mon.0) 1330 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:39:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[31607]: audit 2024-09-10T18:39:28.384703+0000 mon.a (mon.0) 1331 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:29 smithi100 bash[31607]: audit 2024-09-10T18:39:28.387018+0000 mon.a (mon.0) 1332 : audit [INF] from='client.? 172.21.15.100:0/2855743240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72593-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: cluster 2024-09-10T18:39:29.331963+0000 mgr.y (mgr.24512) 223 : cluster [DBG] pgmap v200: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.391688+0000 mon.a (mon.0) 1333 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.399763+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? 172.21.15.100:0/2855743240' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72593-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: cluster 2024-09-10T18:39:29.400152+0000 mon.a (mon.0) 1335 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.414266+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.100:0/1060389643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.415209+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.467860+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.468372+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.469444+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.469793+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.470367+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:30.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:29.470752+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:30.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:30.230424+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:39:30.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:30.231603+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:30 smithi151 bash[22565]: audit 2024-09-10T18:39:30.232217+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: cluster 2024-09-10T18:39:29.331963+0000 mgr.y (mgr.24512) 223 : cluster [DBG] pgmap v200: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.391688+0000 mon.a (mon.0) 1333 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.399763+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? 172.21.15.100:0/2855743240' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72593-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: cluster 2024-09-10T18:39:29.400152+0000 mon.a (mon.0) 1335 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.414266+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.100:0/1060389643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.415209+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.467860+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.468372+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.469444+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.469793+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.470367+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:29.470752+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:30.230424+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:39:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:30.231603+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[18597]: audit 2024-09-10T18:39:30.232217+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: cluster 2024-09-10T18:39:29.331963+0000 mgr.y (mgr.24512) 223 : cluster [DBG] pgmap v200: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.391688+0000 mon.a (mon.0) 1333 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.399763+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? 172.21.15.100:0/2855743240' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72593-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: cluster 2024-09-10T18:39:29.400152+0000 mon.a (mon.0) 1335 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.414266+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.100:0/1060389643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.415209+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.467860+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.468372+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.469444+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.469793+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.470367+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:29.470752+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:30.230424+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:39:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:30.231603+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:30.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:30 smithi100 bash[31607]: audit 2024-09-10T18:39:30.232217+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.393091+0000 mon.a (mon.0) 1341 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.400935+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.401165+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.401266+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: cluster 2024-09-10T18:39:30.401357+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.405018+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.405678+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.406832+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2"}]: dispatch 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.407400+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: audit 2024-09-10T18:39:30.407920+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:31.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: cluster 2024-09-10T18:39:30.504961+0000 osd.4 (osd.4) 14 : cluster [DBG] 96.1 scrub starts 2024-09-10T18:39:31.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:31 smithi151 bash[22565]: cluster 2024-09-10T18:39:30.512026+0000 osd.4 (osd.4) 15 : cluster [DBG] 96.1 scrub ok 2024-09-10T18:39:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:31] "GET /metrics HTTP/1.1" 200 235529 "" "Prometheus/2.43.0" 2024-09-10T18:39:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.393091+0000 mon.a (mon.0) 1341 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.400935+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.401165+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.401266+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: cluster 2024-09-10T18:39:30.401357+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.405018+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.405678+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.406832+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2"}]: dispatch 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.407400+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: audit 2024-09-10T18:39:30.407920+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: cluster 2024-09-10T18:39:30.504961+0000 osd.4 (osd.4) 14 : cluster [DBG] 96.1 scrub starts 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[18597]: cluster 2024-09-10T18:39:30.512026+0000 osd.4 (osd.4) 15 : cluster [DBG] 96.1 scrub ok 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.393091+0000 mon.a (mon.0) 1341 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.400935+0000 mon.a (mon.0) 1342 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-72601-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.401165+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.401266+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: cluster 2024-09-10T18:39:30.401357+0000 mon.a (mon.0) 1345 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.405018+0000 mon.c (mon.2) 216 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.405678+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.406832+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2"}]: dispatch 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.407400+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: audit 2024-09-10T18:39:30.407920+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: cluster 2024-09-10T18:39:30.504961+0000 osd.4 (osd.4) 14 : cluster [DBG] 96.1 scrub starts 2024-09-10T18:39:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:31 smithi100 bash[31607]: cluster 2024-09-10T18:39:30.512026+0000 osd.4 (osd.4) 15 : cluster [DBG] 96.1 scrub ok 2024-09-10T18:39:32.391 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:39:32.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: cluster 2024-09-10T18:39:31.332972+0000 mgr.y (mgr.24512) 224 : cluster [DBG] pgmap v203: 324 pgs: 32 creating+peering, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: audit 2024-09-10T18:39:31.397789+0000 mon.a (mon.0) 1348 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: audit 2024-09-10T18:39:31.819332+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: cluster 2024-09-10T18:39:31.819745+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: audit 2024-09-10T18:39:31.821544+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: audit 2024-09-10T18:39:31.824931+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.100:0/3396242724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: audit 2024-09-10T18:39:31.831254+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: audit 2024-09-10T18:39:31.831550+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:32.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:32 smithi151 bash[22565]: audit 2024-09-10T18:39:31.831811+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: cluster 2024-09-10T18:39:31.332972+0000 mgr.y (mgr.24512) 224 : cluster [DBG] pgmap v203: 324 pgs: 32 creating+peering, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: audit 2024-09-10T18:39:31.397789+0000 mon.a (mon.0) 1348 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: audit 2024-09-10T18:39:31.819332+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: cluster 2024-09-10T18:39:31.819745+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:39:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: audit 2024-09-10T18:39:31.821544+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: audit 2024-09-10T18:39:31.824931+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.100:0/3396242724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: audit 2024-09-10T18:39:31.831254+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: audit 2024-09-10T18:39:31.831550+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[18597]: audit 2024-09-10T18:39:31.831811+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: cluster 2024-09-10T18:39:31.332972+0000 mgr.y (mgr.24512) 224 : cluster [DBG] pgmap v203: 324 pgs: 32 creating+peering, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: audit 2024-09-10T18:39:31.397789+0000 mon.a (mon.0) 1348 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: audit 2024-09-10T18:39:31.819332+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-72746-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: cluster 2024-09-10T18:39:31.819745+0000 mon.a (mon.0) 1350 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: audit 2024-09-10T18:39:31.821544+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: audit 2024-09-10T18:39:31.824931+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.100:0/3396242724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: audit 2024-09-10T18:39:31.831254+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: audit 2024-09-10T18:39:31.831550+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:32.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:32 smithi100 bash[31607]: audit 2024-09-10T18:39:31.831811+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.123836+0000 mgr.y (mgr.24512) 225 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.402102+0000 mon.a (mon.0) 1353 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.415596+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.416129+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.416296+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: cluster 2024-09-10T18:39:32.416477+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.425519+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.100:0/765761752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.430364+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.431135+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:39:33.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:33 smithi151 bash[22565]: audit 2024-09-10T18:39:32.434090+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:39:33.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.123836+0000 mgr.y (mgr.24512) 225 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.402102+0000 mon.a (mon.0) 1353 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.415596+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.416129+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.416296+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: cluster 2024-09-10T18:39:32.416477+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.425519+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.100:0/765761752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.430364+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.431135+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:39:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[31607]: audit 2024-09-10T18:39:32.434090+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.123836+0000 mgr.y (mgr.24512) 225 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.402102+0000 mon.a (mon.0) 1353 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.415596+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-72806-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.416129+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-72593-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.416296+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: cluster 2024-09-10T18:39:32.416477+0000 mon.a (mon.0) 1357 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.425519+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.100:0/765761752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.430364+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.431135+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:39:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:33 smithi100 bash[18597]: audit 2024-09-10T18:39:32.434090+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:39:34.677 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: cluster 2024-09-10T18:39:33.333969+0000 mgr.y (mgr.24512) 226 : cluster [DBG] pgmap v206: 364 pgs: 72 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: audit 2024-09-10T18:39:33.403517+0000 mon.a (mon.0) 1360 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: cluster 2024-09-10T18:39:33.408790+0000 mon.a (mon.0) 1361 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: audit 2024-09-10T18:39:33.416873+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: audit 2024-09-10T18:39:33.417003+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: cluster 2024-09-10T18:39:33.417530+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: audit 2024-09-10T18:39:33.430420+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:39:34.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:34 smithi151 bash[22565]: audit 2024-09-10T18:39:33.430966+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: cluster 2024-09-10T18:39:33.333969+0000 mgr.y (mgr.24512) 226 : cluster [DBG] pgmap v206: 364 pgs: 72 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: audit 2024-09-10T18:39:33.403517+0000 mon.a (mon.0) 1360 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: cluster 2024-09-10T18:39:33.408790+0000 mon.a (mon.0) 1361 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: audit 2024-09-10T18:39:33.416873+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: audit 2024-09-10T18:39:33.417003+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: cluster 2024-09-10T18:39:33.417530+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: audit 2024-09-10T18:39:33.430420+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:39:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[18597]: audit 2024-09-10T18:39:33.430966+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: cluster 2024-09-10T18:39:33.333969+0000 mgr.y (mgr.24512) 226 : cluster [DBG] pgmap v206: 364 pgs: 72 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: audit 2024-09-10T18:39:33.403517+0000 mon.a (mon.0) 1360 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: cluster 2024-09-10T18:39:33.408790+0000 mon.a (mon.0) 1361 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: audit 2024-09-10T18:39:33.416873+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: audit 2024-09-10T18:39:33.417003+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: cluster 2024-09-10T18:39:33.417530+0000 mon.a (mon.0) 1364 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: audit 2024-09-10T18:39:33.430420+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.100:0/1225907398' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:39:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:34 smithi100 bash[31607]: audit 2024-09-10T18:39:33.430966+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:39:35.448 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-10T18:39:35.448 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-10T18:39:35.448 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: seed 72746 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-10T18:39:35.449 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (1254 ms) 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (9 ms) 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (8 ms) 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (7 ms) 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-09-10T18:39:35.450 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (5 ms) 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (8 ms) 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (4 ms) 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-10T18:39:35.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (7 ms) 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (2792 ms) 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (464 ms) 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: done waiting, doing copies 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-10T18:39:35.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: done waiting 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60862 ms) 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (9 ms) 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (5 ms) 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4218 ms) 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-10T18:39:35.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-10T18:39:35.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-10T18:39:35.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-10T18:39:35.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:35 smithi151 bash[22565]: audit 2024-09-10T18:39:34.406936+0000 mon.a (mon.0) 1366 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:35.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:35 smithi151 bash[22565]: audit 2024-09-10T18:39:34.436006+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:39:35.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:35 smithi151 bash[22565]: cluster 2024-09-10T18:39:34.436252+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:39:35.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:35 smithi151 bash[22565]: audit 2024-09-10T18:39:34.446049+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? 172.21.15.100:0/227594631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-72593-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:35.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:35 smithi151 bash[22565]: audit 2024-09-10T18:39:35.357890+0000 mon.c (mon.2) 225 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[31607]: audit 2024-09-10T18:39:34.406936+0000 mon.a (mon.0) 1366 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[31607]: audit 2024-09-10T18:39:34.436006+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:39:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[31607]: cluster 2024-09-10T18:39:34.436252+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:39:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[31607]: audit 2024-09-10T18:39:34.446049+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? 172.21.15.100:0/227594631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-72593-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:35.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[31607]: audit 2024-09-10T18:39:35.357890+0000 mon.c (mon.2) 225 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[18597]: audit 2024-09-10T18:39:34.406936+0000 mon.a (mon.0) 1366 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[18597]: audit 2024-09-10T18:39:34.436006+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-72746-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:39:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[18597]: cluster 2024-09-10T18:39:34.436252+0000 mon.a (mon.0) 1368 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:39:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[18597]: audit 2024-09-10T18:39:34.446049+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? 172.21.15.100:0/227594631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-72593-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:35 smithi100 bash[18597]: audit 2024-09-10T18:39:35.357890+0000 mon.c (mon.2) 225 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: cluster 2024-09-10T18:39:35.336757+0000 mgr.y (mgr.24512) 227 : cluster [DBG] pgmap v209: 332 pgs: 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.408039+0000 mon.a (mon.0) 1370 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.439490+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? 172.21.15.100:0/227594631' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-72593-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: cluster 2024-09-10T18:39:35.439949+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.452089+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? 172.21.15.100:0/158192897' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-72601-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.493756+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.495170+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.495514+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72746-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:36.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.834225+0000 mon.c (mon.2) 226 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:36.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.835892+0000 mon.c (mon.2) 227 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:36.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: cluster 2024-09-10T18:39:35.838824+0000 mgr.y (mgr.24512) 228 : cluster [DBG] pgmap v211: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:36.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:36 smithi151 bash[22565]: audit 2024-09-10T18:39:35.846449+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:39:36.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: cluster 2024-09-10T18:39:35.336757+0000 mgr.y (mgr.24512) 227 : cluster [DBG] pgmap v209: 332 pgs: 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:39:36.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.408039+0000 mon.a (mon.0) 1370 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:36.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.439490+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? 172.21.15.100:0/227594631' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-72593-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:36.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: cluster 2024-09-10T18:39:35.439949+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.452089+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? 172.21.15.100:0/158192897' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-72601-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.493756+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.495170+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.495514+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72746-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.834225+0000 mon.c (mon.2) 226 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.835892+0000 mon.c (mon.2) 227 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: cluster 2024-09-10T18:39:35.838824+0000 mgr.y (mgr.24512) 228 : cluster [DBG] pgmap v211: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[18597]: audit 2024-09-10T18:39:35.846449+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: cluster 2024-09-10T18:39:35.336757+0000 mgr.y (mgr.24512) 227 : cluster [DBG] pgmap v209: 332 pgs: 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.408039+0000 mon.a (mon.0) 1370 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.439490+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? 172.21.15.100:0/227594631' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-72593-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: cluster 2024-09-10T18:39:35.439949+0000 mon.a (mon.0) 1372 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.452089+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? 172.21.15.100:0/158192897' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-72601-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.493756+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.495170+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.495514+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72746-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.834225+0000 mon.c (mon.2) 226 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.835892+0000 mon.c (mon.2) 227 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: cluster 2024-09-10T18:39:35.838824+0000 mgr.y (mgr.24512) 228 : cluster [DBG] pgmap v211: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:36 smithi100 bash[31607]: audit 2024-09-10T18:39:35.846449+0000 mon.a (mon.0) 1377 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:39:37.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:37 smithi151 bash[22565]: audit 2024-09-10T18:39:36.410972+0000 mon.a (mon.0) 1378 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:37.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:37 smithi151 bash[22565]: audit 2024-09-10T18:39:36.441194+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? 172.21.15.100:0/158192897' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-72601-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:37.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:37 smithi151 bash[22565]: audit 2024-09-10T18:39:36.441643+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72746-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:37.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:37 smithi151 bash[22565]: cluster 2024-09-10T18:39:36.441755+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:39:37.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:37 smithi151 bash[22565]: audit 2024-09-10T18:39:36.445019+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72746-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:37.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:37 smithi151 bash[22565]: cluster 2024-09-10T18:39:36.453032+0000 mon.a (mon.0) 1383 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-09-10T18:39:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[18597]: audit 2024-09-10T18:39:36.410972+0000 mon.a (mon.0) 1378 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[18597]: audit 2024-09-10T18:39:36.441194+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? 172.21.15.100:0/158192897' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-72601-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[18597]: audit 2024-09-10T18:39:36.441643+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72746-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[18597]: cluster 2024-09-10T18:39:36.441755+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:39:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[18597]: audit 2024-09-10T18:39:36.445019+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72746-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[18597]: cluster 2024-09-10T18:39:36.453032+0000 mon.a (mon.0) 1383 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-09-10T18:39:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[31607]: audit 2024-09-10T18:39:36.410972+0000 mon.a (mon.0) 1378 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[31607]: audit 2024-09-10T18:39:36.441194+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? 172.21.15.100:0/158192897' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-72601-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[31607]: audit 2024-09-10T18:39:36.441643+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72746-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[31607]: cluster 2024-09-10T18:39:36.441755+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:39:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[31607]: audit 2024-09-10T18:39:36.445019+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72746-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:37 smithi100 bash[31607]: cluster 2024-09-10T18:39:36.453032+0000 mon.a (mon.0) 1383 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-09-10T18:39:38.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:38 smithi151 bash[22565]: audit 2024-09-10T18:39:37.412397+0000 mon.a (mon.0) 1384 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:38.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:38 smithi151 bash[22565]: cluster 2024-09-10T18:39:37.839647+0000 mgr.y (mgr.24512) 229 : cluster [DBG] pgmap v214: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:38.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:38 smithi151 bash[22565]: cluster 2024-09-10T18:39:37.878195+0000 mon.a (mon.0) 1385 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:39:38.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:38 smithi151 bash[22565]: audit 2024-09-10T18:39:37.885248+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.100:0/527785542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-72593-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:38.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[18597]: audit 2024-09-10T18:39:37.412397+0000 mon.a (mon.0) 1384 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:38.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[18597]: cluster 2024-09-10T18:39:37.839647+0000 mgr.y (mgr.24512) 229 : cluster [DBG] pgmap v214: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[18597]: cluster 2024-09-10T18:39:37.878195+0000 mon.a (mon.0) 1385 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:39:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[18597]: audit 2024-09-10T18:39:37.885248+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.100:0/527785542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-72593-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[31607]: audit 2024-09-10T18:39:37.412397+0000 mon.a (mon.0) 1384 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[31607]: cluster 2024-09-10T18:39:37.839647+0000 mgr.y (mgr.24512) 229 : cluster [DBG] pgmap v214: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[31607]: cluster 2024-09-10T18:39:37.878195+0000 mon.a (mon.0) 1385 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:39:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:38 smithi100 bash[31607]: audit 2024-09-10T18:39:37.885248+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? 172.21.15.100:0/527785542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-72593-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:39.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: audit 2024-09-10T18:39:38.414588+0000 mon.a (mon.0) 1387 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: audit 2024-09-10T18:39:38.450734+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72746-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: audit 2024-09-10T18:39:38.451124+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.100:0/527785542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-72593-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: cluster 2024-09-10T18:39:38.451337+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:39:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: audit 2024-09-10T18:39:38.453714+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.100:0/3100616302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: audit 2024-09-10T18:39:38.456100+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: audit 2024-09-10T18:39:39.330617+0000 mon.a (mon.0) 1392 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:39:39.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:39 smithi151 bash[22565]: audit 2024-09-10T18:39:39.332659+0000 mon.c (mon.2) 229 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:39.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: audit 2024-09-10T18:39:38.414588+0000 mon.a (mon.0) 1387 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:39.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: audit 2024-09-10T18:39:38.450734+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72746-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:39.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: audit 2024-09-10T18:39:38.451124+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.100:0/527785542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-72593-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:39.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: cluster 2024-09-10T18:39:38.451337+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:39:39.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: audit 2024-09-10T18:39:38.453714+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.100:0/3100616302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:39.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: audit 2024-09-10T18:39:38.456100+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: audit 2024-09-10T18:39:39.330617+0000 mon.a (mon.0) 1392 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[31607]: audit 2024-09-10T18:39:39.332659+0000 mon.c (mon.2) 229 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: audit 2024-09-10T18:39:38.414588+0000 mon.a (mon.0) 1387 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: audit 2024-09-10T18:39:38.450734+0000 mon.a (mon.0) 1388 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72746-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: audit 2024-09-10T18:39:38.451124+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.100:0/527785542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-72593-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: cluster 2024-09-10T18:39:38.451337+0000 mon.a (mon.0) 1390 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: audit 2024-09-10T18:39:38.453714+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.100:0/3100616302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: audit 2024-09-10T18:39:38.456100+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: audit 2024-09-10T18:39:39.330617+0000 mon.a (mon.0) 1392 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:39:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:39 smithi100 bash[18597]: audit 2024-09-10T18:39:39.332659+0000 mon.c (mon.2) 229 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:40.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:40 smithi151 bash[22565]: audit 2024-09-10T18:39:39.416831+0000 mon.a (mon.0) 1393 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:40.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:40 smithi151 bash[22565]: audit 2024-09-10T18:39:39.457518+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:40.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:40 smithi151 bash[22565]: cluster 2024-09-10T18:39:39.457923+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:39:40.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:40 smithi151 bash[22565]: audit 2024-09-10T18:39:39.464314+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:40.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:40 smithi151 bash[22565]: cluster 2024-09-10T18:39:39.840510+0000 mgr.y (mgr.24512) 230 : cluster [DBG] pgmap v217: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[31607]: audit 2024-09-10T18:39:39.416831+0000 mon.a (mon.0) 1393 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[31607]: audit 2024-09-10T18:39:39.457518+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[31607]: cluster 2024-09-10T18:39:39.457923+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:39:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[31607]: audit 2024-09-10T18:39:39.464314+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[31607]: cluster 2024-09-10T18:39:39.840510+0000 mgr.y (mgr.24512) 230 : cluster [DBG] pgmap v217: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[18597]: audit 2024-09-10T18:39:39.416831+0000 mon.a (mon.0) 1393 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[18597]: audit 2024-09-10T18:39:39.457518+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-72601-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[18597]: cluster 2024-09-10T18:39:39.457923+0000 mon.a (mon.0) 1395 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:39:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[18597]: audit 2024-09-10T18:39:39.464314+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:40 smithi100 bash[18597]: cluster 2024-09-10T18:39:39.840510+0000 mgr.y (mgr.24512) 230 : cluster [DBG] pgmap v217: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:39:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:41 smithi151 bash[22565]: audit 2024-09-10T18:39:40.422006+0000 mon.a (mon.0) 1397 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:41 smithi151 bash[22565]: cluster 2024-09-10T18:39:40.450165+0000 mon.a (mon.0) 1398 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:41 smithi151 bash[22565]: audit 2024-09-10T18:39:40.458570+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.100:0/1849879961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:41 smithi151 bash[22565]: audit 2024-09-10T18:39:40.467209+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:41 smithi151 bash[22565]: cluster 2024-09-10T18:39:40.467737+0000 mon.a (mon.0) 1400 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:39:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:41 smithi151 bash[22565]: audit 2024-09-10T18:39:40.479242+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:41 smithi151 bash[22565]: audit 2024-09-10T18:39:40.481701+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:41] "GET /metrics HTTP/1.1" 200 240505 "" "Prometheus/2.43.0" 2024-09-10T18:39:41.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18597]: audit 2024-09-10T18:39:40.422006+0000 mon.a (mon.0) 1397 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:41.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18597]: cluster 2024-09-10T18:39:40.450165+0000 mon.a (mon.0) 1398 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18597]: audit 2024-09-10T18:39:40.458570+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.100:0/1849879961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18597]: audit 2024-09-10T18:39:40.467209+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18597]: cluster 2024-09-10T18:39:40.467737+0000 mon.a (mon.0) 1400 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18597]: audit 2024-09-10T18:39:40.479242+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[18597]: audit 2024-09-10T18:39:40.481701+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[31607]: audit 2024-09-10T18:39:40.422006+0000 mon.a (mon.0) 1397 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[31607]: cluster 2024-09-10T18:39:40.450165+0000 mon.a (mon.0) 1398 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[31607]: audit 2024-09-10T18:39:40.458570+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.100:0/1849879961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[31607]: audit 2024-09-10T18:39:40.467209+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[31607]: cluster 2024-09-10T18:39:40.467737+0000 mon.a (mon.0) 1400 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:39:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[31607]: audit 2024-09-10T18:39:40.479242+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:41 smithi100 bash[31607]: audit 2024-09-10T18:39:40.481701+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:42.417 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:39:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[22565]: audit 2024-09-10T18:39:41.423482+0000 mon.a (mon.0) 1403 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[22565]: audit 2024-09-10T18:39:41.473170+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[22565]: audit 2024-09-10T18:39:41.473477+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[22565]: cluster 2024-09-10T18:39:41.473724+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:39:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[22565]: audit 2024-09-10T18:39:41.486442+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.100:0/2091696798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[22565]: audit 2024-09-10T18:39:41.487216+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:42.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:42 smithi151 bash[22565]: cluster 2024-09-10T18:39:41.841559+0000 mgr.y (mgr.24512) 231 : cluster [DBG] pgmap v220: 372 pgs: 32 creating+peering, 64 unknown, 276 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:42.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[18597]: audit 2024-09-10T18:39:41.423482+0000 mon.a (mon.0) 1403 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:42.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[18597]: audit 2024-09-10T18:39:41.473170+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:42.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[18597]: audit 2024-09-10T18:39:41.473477+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:42.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[18597]: cluster 2024-09-10T18:39:41.473724+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:39:42.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[18597]: audit 2024-09-10T18:39:41.486442+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.100:0/2091696798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:42.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[18597]: audit 2024-09-10T18:39:41.487216+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:42.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[18597]: cluster 2024-09-10T18:39:41.841559+0000 mgr.y (mgr.24512) 231 : cluster [DBG] pgmap v220: 372 pgs: 32 creating+peering, 64 unknown, 276 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:42.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[31607]: audit 2024-09-10T18:39:41.423482+0000 mon.a (mon.0) 1403 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:42.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[31607]: audit 2024-09-10T18:39:41.473170+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-72593-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:42.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[31607]: audit 2024-09-10T18:39:41.473477+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72746-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:42.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[31607]: cluster 2024-09-10T18:39:41.473724+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:39:42.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[31607]: audit 2024-09-10T18:39:41.486442+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.100:0/2091696798' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:42.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[31607]: audit 2024-09-10T18:39:41.487216+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:42.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:42 smithi100 bash[31607]: cluster 2024-09-10T18:39:41.841559+0000 mgr.y (mgr.24512) 231 : cluster [DBG] pgmap v220: 372 pgs: 32 creating+peering, 64 unknown, 276 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:39:43.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:43 smithi151 bash[22565]: audit 2024-09-10T18:39:42.135535+0000 mgr.y (mgr.24512) 232 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:43.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:43 smithi151 bash[22565]: audit 2024-09-10T18:39:42.425202+0000 mon.a (mon.0) 1408 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:43.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:43 smithi151 bash[22565]: audit 2024-09-10T18:39:42.477141+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:43.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:43 smithi151 bash[22565]: cluster 2024-09-10T18:39:42.477513+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:39:43.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:43 smithi151 bash[22565]: audit 2024-09-10T18:39:42.484283+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:43.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:43 smithi151 bash[22565]: audit 2024-09-10T18:39:42.494856+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[18597]: audit 2024-09-10T18:39:42.135535+0000 mgr.y (mgr.24512) 232 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[18597]: audit 2024-09-10T18:39:42.425202+0000 mon.a (mon.0) 1408 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[18597]: audit 2024-09-10T18:39:42.477141+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[18597]: cluster 2024-09-10T18:39:42.477513+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:39:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[18597]: audit 2024-09-10T18:39:42.484283+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[18597]: audit 2024-09-10T18:39:42.494856+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[31607]: audit 2024-09-10T18:39:42.135535+0000 mgr.y (mgr.24512) 232 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[31607]: audit 2024-09-10T18:39:42.425202+0000 mon.a (mon.0) 1408 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[31607]: audit 2024-09-10T18:39:42.477141+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-72601-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[31607]: cluster 2024-09-10T18:39:42.477513+0000 mon.a (mon.0) 1410 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:39:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[31607]: audit 2024-09-10T18:39:42.484283+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:43 smithi100 bash[31607]: audit 2024-09-10T18:39:42.494856+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (6 api_snapshots_pp: Running main() from gmock_main.cc 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: seed 72806 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2359 ms) 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (3186 ms) 2024-09-10T18:39:44.496 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2773 ms) 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2378 ms) 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3837 ms) 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (14533 ms total) 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (3917 ms) 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4723 ms) 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-10T18:39:44.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6333 ms) 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4164 ms) 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2032 ms) 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi100-72806-7 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17991 ms) 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (39160 ms total) 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-10T18:39:44.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5123 ms) 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5132 ms total) 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3024 ms) 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2429 ms) 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (1583 ms) 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-10T18:39:44.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2025 ms) 2024-09-10T18:39:44.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9061 ms total) 2024-09-10T18:39:44.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-09-10T18:39:44.677 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: audit 2024-09-10T18:39:43.428133+0000 mon.a (mon.0) 1412 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: audit 2024-09-10T18:39:43.479392+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.100:0/4014939347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: audit 2024-09-10T18:39:43.486619+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: cluster 2024-09-10T18:39:43.486990+0000 mon.a (mon.0) 1414 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: audit 2024-09-10T18:39:43.491016+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: audit 2024-09-10T18:39:43.496108+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: audit 2024-09-10T18:39:43.496527+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: audit 2024-09-10T18:39:43.496885+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:44.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:44 smithi151 bash[22565]: cluster 2024-09-10T18:39:43.842402+0000 mgr.y (mgr.24512) 233 : cluster [DBG] pgmap v223: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:44.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: audit 2024-09-10T18:39:43.428133+0000 mon.a (mon.0) 1412 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:44.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: audit 2024-09-10T18:39:43.479392+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.100:0/4014939347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:44.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: audit 2024-09-10T18:39:43.486619+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:44.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: cluster 2024-09-10T18:39:43.486990+0000 mon.a (mon.0) 1414 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: audit 2024-09-10T18:39:43.491016+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: audit 2024-09-10T18:39:43.496108+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: audit 2024-09-10T18:39:43.496527+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: audit 2024-09-10T18:39:43.496885+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[18597]: cluster 2024-09-10T18:39:43.842402+0000 mgr.y (mgr.24512) 233 : cluster [DBG] pgmap v223: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: audit 2024-09-10T18:39:43.428133+0000 mon.a (mon.0) 1412 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: audit 2024-09-10T18:39:43.479392+0000 mon.b (mon.1) 102 : audit [INF] from='client.? 172.21.15.100:0/4014939347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: audit 2024-09-10T18:39:43.486619+0000 mon.a (mon.0) 1413 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: cluster 2024-09-10T18:39:43.486990+0000 mon.a (mon.0) 1414 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: audit 2024-09-10T18:39:43.491016+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.100:0/1206179562' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: audit 2024-09-10T18:39:43.496108+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]: dispatch 2024-09-10T18:39:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: audit 2024-09-10T18:39:43.496527+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: audit 2024-09-10T18:39:43.496885+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:44 smithi100 bash[31607]: cluster 2024-09-10T18:39:43.842402+0000 mgr.y (mgr.24512) 233 : cluster [DBG] pgmap v223: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:45.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.431504+0000 mon.a (mon.0) 1418 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:45.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.489942+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.490111+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.490303+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: cluster 2024-09-10T18:39:44.490660+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.494489+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.100:0/1130866853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.495819+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.496361+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.528929+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.529267+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.530071+0000 mon.c (mon.2) 235 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.530404+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.530914+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:45.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:45 smithi151 bash[22565]: audit 2024-09-10T18:39:44.531185+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:45.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.431504+0000 mon.a (mon.0) 1418 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:45.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.489942+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:45.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.490111+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.490303+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: cluster 2024-09-10T18:39:44.490660+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.494489+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.100:0/1130866853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.495819+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.496361+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.528929+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.529267+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.530071+0000 mon.c (mon.2) 235 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.530404+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.530914+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[31607]: audit 2024-09-10T18:39:44.531185+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.431504+0000 mon.a (mon.0) 1418 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.489942+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-72806-15"}]': finished 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.490111+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.490303+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-72593-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: cluster 2024-09-10T18:39:44.490660+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.494489+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.100:0/1130866853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.495819+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]: dispatch 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.496361+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.528929+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.529267+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.530071+0000 mon.c (mon.2) 235 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.530404+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:45.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.530914+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:45.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:45 smithi100 bash[18597]: audit 2024-09-10T18:39:44.531185+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.434502+0000 mon.a (mon.0) 1428 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.493978+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.494201+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.494778+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: cluster 2024-09-10T18:39:45.494967+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:39:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.497100+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:46.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.501258+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.523650+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: cluster 2024-09-10T18:39:45.843218+0000 mgr.y (mgr.24512) 234 : cluster [DBG] pgmap v226: 292 pgs: 32 creating+peering, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:39:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.944142+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.946561+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.958251+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.961046+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:46 smithi151 bash[22565]: audit 2024-09-10T18:39:45.963092+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:46.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.434502+0000 mon.a (mon.0) 1428 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:46.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.493978+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.494201+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.494778+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: cluster 2024-09-10T18:39:45.494967+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.497100+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.501258+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.523650+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: cluster 2024-09-10T18:39:45.843218+0000 mgr.y (mgr.24512) 234 : cluster [DBG] pgmap v226: 292 pgs: 32 creating+peering, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.944142+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.946561+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.958251+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.961046+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[18597]: audit 2024-09-10T18:39:45.963092+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.434502+0000 mon.a (mon.0) 1428 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.493978+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? 172.21.15.100:0/204609288' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72746-24"}]': finished 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.494201+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-72601-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.494778+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: cluster 2024-09-10T18:39:45.494967+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.497100+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.501258+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.523650+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: cluster 2024-09-10T18:39:45.843218+0000 mgr.y (mgr.24512) 234 : cluster [DBG] pgmap v226: 292 pgs: 32 creating+peering, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:39:46.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.944142+0000 mon.b (mon.1) 104 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.946561+0000 mon.b (mon.1) 105 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:46.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.958251+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.961046+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:46.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:46 smithi100 bash[31607]: audit 2024-09-10T18:39:45.963092+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:47.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[31607]: audit 2024-09-10T18:39:46.436928+0000 mon.a (mon.0) 1437 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:47.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[31607]: cluster 2024-09-10T18:39:46.489478+0000 mon.a (mon.0) 1438 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:47.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[31607]: audit 2024-09-10T18:39:46.918451+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[31607]: audit 2024-09-10T18:39:46.932131+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[31607]: cluster 2024-09-10T18:39:46.932207+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[31607]: audit 2024-09-10T18:39:46.935716+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? 172.21.15.100:0/3835627392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-72746-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[31607]: audit 2024-09-10T18:39:46.943962+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[18597]: audit 2024-09-10T18:39:46.436928+0000 mon.a (mon.0) 1437 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[18597]: cluster 2024-09-10T18:39:46.489478+0000 mon.a (mon.0) 1438 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[18597]: audit 2024-09-10T18:39:46.918451+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[18597]: audit 2024-09-10T18:39:46.932131+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[18597]: cluster 2024-09-10T18:39:46.932207+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:39:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[18597]: audit 2024-09-10T18:39:46.935716+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? 172.21.15.100:0/3835627392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-72746-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:47 smithi100 bash[18597]: audit 2024-09-10T18:39:46.943962+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:47.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:47 smithi151 bash[22565]: audit 2024-09-10T18:39:46.436928+0000 mon.a (mon.0) 1437 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:47.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:47 smithi151 bash[22565]: cluster 2024-09-10T18:39:46.489478+0000 mon.a (mon.0) 1438 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:47.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:47 smithi151 bash[22565]: audit 2024-09-10T18:39:46.918451+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:47.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:47 smithi151 bash[22565]: audit 2024-09-10T18:39:46.932131+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-72593-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:47.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:47 smithi151 bash[22565]: cluster 2024-09-10T18:39:46.932207+0000 mon.a (mon.0) 1440 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:39:47.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:47 smithi151 bash[22565]: audit 2024-09-10T18:39:46.935716+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? 172.21.15.100:0/3835627392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-72746-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:47.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:47 smithi151 bash[22565]: audit 2024-09-10T18:39:46.943962+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:48.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[31607]: audit 2024-09-10T18:39:47.437986+0000 mon.a (mon.0) 1443 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:48.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[31607]: audit 2024-09-10T18:39:47.501422+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:39:48.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[31607]: audit 2024-09-10T18:39:47.501843+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.100:0/3835627392' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-72746-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:48.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[31607]: cluster 2024-09-10T18:39:47.844216+0000 mgr.y (mgr.24512) 235 : cluster [DBG] pgmap v229: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[31607]: cluster 2024-09-10T18:39:47.924337+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[31607]: audit 2024-09-10T18:39:47.926555+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.100:0/3242915028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[31607]: audit 2024-09-10T18:39:47.934751+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[18597]: audit 2024-09-10T18:39:47.437986+0000 mon.a (mon.0) 1443 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[18597]: audit 2024-09-10T18:39:47.501422+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[18597]: audit 2024-09-10T18:39:47.501843+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.100:0/3835627392' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-72746-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[18597]: cluster 2024-09-10T18:39:47.844216+0000 mgr.y (mgr.24512) 235 : cluster [DBG] pgmap v229: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[18597]: cluster 2024-09-10T18:39:47.924337+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[18597]: audit 2024-09-10T18:39:47.926555+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.100:0/3242915028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:48 smithi100 bash[18597]: audit 2024-09-10T18:39:47.934751+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:48.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:48 smithi151 bash[22565]: audit 2024-09-10T18:39:47.437986+0000 mon.a (mon.0) 1443 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:48.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:48 smithi151 bash[22565]: audit 2024-09-10T18:39:47.501422+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:39:48.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:48 smithi151 bash[22565]: audit 2024-09-10T18:39:47.501843+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? 172.21.15.100:0/3835627392' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-72746-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:48.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:48 smithi151 bash[22565]: cluster 2024-09-10T18:39:47.844216+0000 mgr.y (mgr.24512) 235 : cluster [DBG] pgmap v229: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:48.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:48 smithi151 bash[22565]: cluster 2024-09-10T18:39:47.924337+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:39:48.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:48 smithi151 bash[22565]: audit 2024-09-10T18:39:47.926555+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.100:0/3242915028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:48.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:48 smithi151 bash[22565]: audit 2024-09-10T18:39:47.934751+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:49.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[18597]: audit 2024-09-10T18:39:48.441251+0000 mon.a (mon.0) 1448 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:49.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[18597]: audit 2024-09-10T18:39:48.511087+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:49.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[18597]: audit 2024-09-10T18:39:48.511409+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:49.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[18597]: cluster 2024-09-10T18:39:48.511518+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:39:49.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[31607]: audit 2024-09-10T18:39:48.441251+0000 mon.a (mon.0) 1448 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[31607]: audit 2024-09-10T18:39:48.511087+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[31607]: audit 2024-09-10T18:39:48.511409+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:49 smithi100 bash[31607]: cluster 2024-09-10T18:39:48.511518+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:39:49.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:49 smithi151 bash[22565]: audit 2024-09-10T18:39:48.441251+0000 mon.a (mon.0) 1448 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:49.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:49 smithi151 bash[22565]: audit 2024-09-10T18:39:48.511087+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-72593-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:49.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:49 smithi151 bash[22565]: audit 2024-09-10T18:39:48.511409+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-72601-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:49.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:49 smithi151 bash[22565]: cluster 2024-09-10T18:39:48.511518+0000 mon.a (mon.0) 1451 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:39:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[18597]: audit 2024-09-10T18:39:49.442587+0000 mon.a (mon.0) 1452 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[18597]: cluster 2024-09-10T18:39:49.519286+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:39:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[18597]: audit 2024-09-10T18:39:49.534961+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? 172.21.15.100:0/2790607484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-72746-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[18597]: cluster 2024-09-10T18:39:49.845123+0000 mgr.y (mgr.24512) 236 : cluster [DBG] pgmap v232: 308 pgs: 48 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[31607]: audit 2024-09-10T18:39:49.442587+0000 mon.a (mon.0) 1452 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[31607]: cluster 2024-09-10T18:39:49.519286+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:39:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[31607]: audit 2024-09-10T18:39:49.534961+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? 172.21.15.100:0/2790607484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-72746-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:50 smithi100 bash[31607]: cluster 2024-09-10T18:39:49.845123+0000 mgr.y (mgr.24512) 236 : cluster [DBG] pgmap v232: 308 pgs: 48 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:50 smithi151 bash[22565]: audit 2024-09-10T18:39:49.442587+0000 mon.a (mon.0) 1452 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:50.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:50 smithi151 bash[22565]: cluster 2024-09-10T18:39:49.519286+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:39:50.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:50 smithi151 bash[22565]: audit 2024-09-10T18:39:49.534961+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? 172.21.15.100:0/2790607484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-72746-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:50.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:50 smithi151 bash[22565]: cluster 2024-09-10T18:39:49.845123+0000 mgr.y (mgr.24512) 236 : cluster [DBG] pgmap v232: 308 pgs: 48 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:51] "GET /metrics HTTP/1.1" 200 240505 "" "Prometheus/2.43.0" 2024-09-10T18:39:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[31607]: audit 2024-09-10T18:39:50.444675+0000 mon.a (mon.0) 1455 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[31607]: audit 2024-09-10T18:39:50.511615+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[31607]: audit 2024-09-10T18:39:50.524193+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? 172.21.15.100:0/2790607484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-72746-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[31607]: cluster 2024-09-10T18:39:50.524624+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:39:51.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[31607]: audit 2024-09-10T18:39:50.543079+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:51.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[31607]: audit 2024-09-10T18:39:50.543291+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? 172.21.15.100:0/973809284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-72601-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:51.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[18597]: audit 2024-09-10T18:39:50.444675+0000 mon.a (mon.0) 1455 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:51.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[18597]: audit 2024-09-10T18:39:50.511615+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:51.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[18597]: audit 2024-09-10T18:39:50.524193+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? 172.21.15.100:0/2790607484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-72746-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:51.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[18597]: cluster 2024-09-10T18:39:50.524624+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:39:51.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[18597]: audit 2024-09-10T18:39:50.543079+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:51.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:51 smithi100 bash[18597]: audit 2024-09-10T18:39:50.543291+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? 172.21.15.100:0/973809284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-72601-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:51.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:51 smithi151 bash[22565]: audit 2024-09-10T18:39:50.444675+0000 mon.a (mon.0) 1455 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:51.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:51 smithi151 bash[22565]: audit 2024-09-10T18:39:50.511615+0000 mon.b (mon.1) 107 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:51.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:51 smithi151 bash[22565]: audit 2024-09-10T18:39:50.524193+0000 mon.a (mon.0) 1456 : audit [INF] from='client.? 172.21.15.100:0/2790607484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-72746-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:51.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:51 smithi151 bash[22565]: cluster 2024-09-10T18:39:50.524624+0000 mon.a (mon.0) 1457 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:39:51.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:51 smithi151 bash[22565]: audit 2024-09-10T18:39:50.543079+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:51.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:51 smithi151 bash[22565]: audit 2024-09-10T18:39:50.543291+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? 172.21.15.100:0/973809284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-72601-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:52.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:39:52.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-09-10T18:39:52.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-09-10T18:39:52.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] Global test environment set-up. 2024-09-10T18:39:52.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-09-10T18:39:52.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.TooBig (10497 ms) 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (4841 ms) 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (6014 ms) 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3689 ms) 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-10T18:39:52.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (2505 ms) 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3258 ms) 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3205 ms) 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (4209 ms) 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3618 ms) 2024-09-10T18:39:52.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3301 ms) 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (4144 ms) 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3006 ms) 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3012 ms) 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2716 ms) 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-10T18:39:52.551 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.Flush (2989 ms) 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3028 ms) 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2976 ms) 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3022 ms) 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3065 ms) 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3020 ms) 2024-09-10T18:39:52.552 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3026 ms) 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3022 ms) 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3012 ms) 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3019 ms) 2024-09-10T18:39:52.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-10T18:39:52.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3014 ms) 2024-09-10T18:39:52.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 25 tests from LibRadosAio (91208 ms total) 2024-09-10T18:39:52.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: 2024-09-10T18:39:52.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-10T18:39:52.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-10T18:39:52.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7044 ms) 2024-09-10T18:39:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[31607]: audit 2024-09-10T18:39:51.446393+0000 mon.a (mon.0) 1460 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[31607]: audit 2024-09-10T18:39:51.511392+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[31607]: audit 2024-09-10T18:39:51.525308+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[31607]: audit 2024-09-10T18:39:51.525514+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? 172.21.15.100:0/973809284' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-72601-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[31607]: cluster 2024-09-10T18:39:51.525751+0000 mon.a (mon.0) 1463 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:39:52.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[31607]: audit 2024-09-10T18:39:51.530591+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[31607]: cluster 2024-09-10T18:39:51.845999+0000 mgr.y (mgr.24512) 237 : cluster [DBG] pgmap v235: 300 pgs: 32 unknown, 268 active+clean; 608 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[18597]: audit 2024-09-10T18:39:51.446393+0000 mon.a (mon.0) 1460 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[18597]: audit 2024-09-10T18:39:51.511392+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[18597]: audit 2024-09-10T18:39:51.525308+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[18597]: audit 2024-09-10T18:39:51.525514+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? 172.21.15.100:0/973809284' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-72601-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[18597]: cluster 2024-09-10T18:39:51.525751+0000 mon.a (mon.0) 1463 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[18597]: audit 2024-09-10T18:39:51.530591+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:52 smithi100 bash[18597]: cluster 2024-09-10T18:39:51.845999+0000 mgr.y (mgr.24512) 237 : cluster [DBG] pgmap v235: 300 pgs: 32 unknown, 268 active+clean; 608 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T18:39:52.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[22565]: audit 2024-09-10T18:39:51.446393+0000 mon.a (mon.0) 1460 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:52.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[22565]: audit 2024-09-10T18:39:51.511392+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.100:0/92427885' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:52.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[22565]: audit 2024-09-10T18:39:51.525308+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:52.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[22565]: audit 2024-09-10T18:39:51.525514+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? 172.21.15.100:0/973809284' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-72601-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:52.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[22565]: cluster 2024-09-10T18:39:51.525751+0000 mon.a (mon.0) 1463 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:39:52.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[22565]: audit 2024-09-10T18:39:51.530591+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]: dispatch 2024-09-10T18:39:52.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:52 smithi151 bash[22565]: cluster 2024-09-10T18:39:51.845999+0000 mgr.y (mgr.24512) 237 : cluster [DBG] pgmap v235: 300 pgs: 32 unknown, 268 active+clean; 608 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T18:39:53.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.143547+0000 mgr.y (mgr.24512) 238 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:53.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.450352+0000 mon.a (mon.0) 1465 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:53.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: cluster 2024-09-10T18:39:52.521204+0000 mon.a (mon.0) 1466 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.532413+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: cluster 2024-09-10T18:39:52.532806+0000 mon.a (mon.0) 1468 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.539265+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.100:0/1331012584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.545510+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.577300+0000 mon.c (mon.2) 240 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.578206+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.581605+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.582042+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.582939+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[18597]: audit 2024-09-10T18:39:52.583603+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.143547+0000 mgr.y (mgr.24512) 238 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.450352+0000 mon.a (mon.0) 1465 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: cluster 2024-09-10T18:39:52.521204+0000 mon.a (mon.0) 1466 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.532413+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: cluster 2024-09-10T18:39:52.532806+0000 mon.a (mon.0) 1468 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.539265+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.100:0/1331012584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.545510+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:53.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.577300+0000 mon.c (mon.2) 240 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.578206+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.581605+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.582042+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.582939+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:53.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:53 smithi100 bash[31607]: audit 2024-09-10T18:39:52.583603+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.143547+0000 mgr.y (mgr.24512) 238 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.450352+0000 mon.a (mon.0) 1465 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: cluster 2024-09-10T18:39:52.521204+0000 mon.a (mon.0) 1466 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.532413+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-72593-26"}]': finished 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: cluster 2024-09-10T18:39:52.532806+0000 mon.a (mon.0) 1468 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.539265+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.100:0/1331012584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.545510+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.577300+0000 mon.c (mon.2) 240 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.578206+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.581605+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.582042+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:53.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.582939+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:53.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:53 smithi151 bash[22565]: audit 2024-09-10T18:39:52.583603+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:53.453410+0000 mon.a (mon.0) 1473 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:53.534952+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:53.535561+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:54.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: cluster 2024-09-10T18:39:53.535827+0000 mon.a (mon.0) 1476 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:39:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:53.539648+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:53.541321+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.100:0/2265228476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:53.551335+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:53.552408+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: cluster 2024-09-10T18:39:53.846911+0000 mgr.y (mgr.24512) 239 : cluster [DBG] pgmap v238: 332 pgs: 64 unknown, 268 active+clean; 608 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T18:39:54.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[31607]: audit 2024-09-10T18:39:54.324426+0000 mon.c (mon.2) 245 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:53.453410+0000 mon.a (mon.0) 1473 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:53.534952+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:53.535561+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: cluster 2024-09-10T18:39:53.535827+0000 mon.a (mon.0) 1476 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:53.539648+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:53.541321+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.100:0/2265228476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:53.551335+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:53.552408+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: cluster 2024-09-10T18:39:53.846911+0000 mgr.y (mgr.24512) 239 : cluster [DBG] pgmap v238: 332 pgs: 64 unknown, 268 active+clean; 608 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T18:39:54.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:54 smithi100 bash[18597]: audit 2024-09-10T18:39:54.324426+0000 mon.c (mon.2) 245 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:54.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:39:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:54.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:53.453410+0000 mon.a (mon.0) 1473 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:54.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:53.534952+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-72746-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:54.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:53.535561+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-72593-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:54.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: cluster 2024-09-10T18:39:53.535827+0000 mon.a (mon.0) 1476 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:39:54.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:53.539648+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:54.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:53.541321+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.100:0/2265228476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:54.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:53.551335+0000 mon.a (mon.0) 1477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:54.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:53.552408+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:54.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: cluster 2024-09-10T18:39:53.846911+0000 mgr.y (mgr.24512) 239 : cluster [DBG] pgmap v238: 332 pgs: 64 unknown, 268 active+clean; 608 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T18:39:54.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:54 smithi151 bash[22565]: audit 2024-09-10T18:39:54.324426+0000 mon.c (mon.2) 245 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:55.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.454763+0000 mon.a (mon.0) 1479 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:55.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.800558+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.975697+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: cluster 2024-09-10T18:39:54.976238+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.990269+0000 mon.a (mon.0) 1482 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.996427+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.996775+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.998273+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.998596+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.999079+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[18597]: audit 2024-09-10T18:39:54.999367+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.454763+0000 mon.a (mon.0) 1479 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.800558+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.975697+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: cluster 2024-09-10T18:39:54.976238+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.990269+0000 mon.a (mon.0) 1482 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.996427+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.996775+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.998273+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.998596+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.999079+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:55 smithi100 bash[31607]: audit 2024-09-10T18:39:54.999367+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:55.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.454763+0000 mon.a (mon.0) 1479 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:55.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.800558+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:55.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.975697+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-72601-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:55.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: cluster 2024-09-10T18:39:54.976238+0000 mon.a (mon.0) 1481 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:39:55.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.990269+0000 mon.a (mon.0) 1482 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:39:55.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.996427+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.996775+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.998273+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.998596+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:55.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.999079+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:55.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:55 smithi151 bash[22565]: audit 2024-09-10T18:39:54.999367+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:39:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.456234+0000 mon.a (mon.0) 1486 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.529590+0000 mon.b (mon.1) 110 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.542653+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:39:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.543008+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.543186+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: cluster 2024-09-10T18:39:55.543351+0000 mon.a (mon.0) 1490 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.548334+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.548447+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: audit 2024-09-10T18:39:55.548788+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[18597]: cluster 2024-09-10T18:39:55.847800+0000 mgr.y (mgr.24512) 240 : cluster [DBG] pgmap v241: 276 pgs: 8 unknown, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.456234+0000 mon.a (mon.0) 1486 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.529590+0000 mon.b (mon.1) 110 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.542653+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:39:56.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.543008+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.543186+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: cluster 2024-09-10T18:39:55.543351+0000 mon.a (mon.0) 1490 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:39:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.548334+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.548447+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: audit 2024-09-10T18:39:55.548788+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:56 smithi100 bash[31607]: cluster 2024-09-10T18:39:55.847800+0000 mgr.y (mgr.24512) 240 : cluster [DBG] pgmap v241: 276 pgs: 8 unknown, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.456234+0000 mon.a (mon.0) 1486 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.529590+0000 mon.b (mon.1) 110 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.542653+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-72593-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.543008+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.543186+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: cluster 2024-09-10T18:39:55.543351+0000 mon.a (mon.0) 1490 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.548334+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:56.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.548447+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]: dispatch 2024-09-10T18:39:56.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: audit 2024-09-10T18:39:55.548788+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:39:56.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:56 smithi151 bash[22565]: cluster 2024-09-10T18:39:55.847800+0000 mgr.y (mgr.24512) 240 : cluster [DBG] pgmap v241: 276 pgs: 8 unknown, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-09-10T18:39:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[18597]: audit 2024-09-10T18:39:56.459757+0000 mon.a (mon.0) 1493 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[18597]: cluster 2024-09-10T18:39:56.537696+0000 mon.a (mon.0) 1494 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[18597]: audit 2024-09-10T18:39:56.552343+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]': finished 2024-09-10T18:39:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[18597]: cluster 2024-09-10T18:39:56.977337+0000 mon.a (mon.0) 1496 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:39:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[18597]: audit 2024-09-10T18:39:56.987566+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? 172.21.15.100:0/3581556795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-72601-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[31607]: audit 2024-09-10T18:39:56.459757+0000 mon.a (mon.0) 1493 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[31607]: cluster 2024-09-10T18:39:56.537696+0000 mon.a (mon.0) 1494 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[31607]: audit 2024-09-10T18:39:56.552343+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]': finished 2024-09-10T18:39:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[31607]: cluster 2024-09-10T18:39:56.977337+0000 mon.a (mon.0) 1496 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:39:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:57 smithi100 bash[31607]: audit 2024-09-10T18:39:56.987566+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? 172.21.15.100:0/3581556795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-72601-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:57.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:57 smithi151 bash[22565]: audit 2024-09-10T18:39:56.459757+0000 mon.a (mon.0) 1493 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:57.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:57 smithi151 bash[22565]: cluster 2024-09-10T18:39:56.537696+0000 mon.a (mon.0) 1494 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:39:57.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:57 smithi151 bash[22565]: audit 2024-09-10T18:39:56.552343+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-13"}]': finished 2024-09-10T18:39:57.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:57 smithi151 bash[22565]: cluster 2024-09-10T18:39:56.977337+0000 mon.a (mon.0) 1496 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:39:57.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:57 smithi151 bash[22565]: audit 2024-09-10T18:39:56.987566+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? 172.21.15.100:0/3581556795' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-72601-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[18597]: audit 2024-09-10T18:39:57.461189+0000 mon.a (mon.0) 1498 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[18597]: audit 2024-09-10T18:39:57.552353+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:39:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[18597]: audit 2024-09-10T18:39:57.552803+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? 172.21.15.100:0/3581556795' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-72601-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[18597]: cluster 2024-09-10T18:39:57.553129+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:39:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[18597]: audit 2024-09-10T18:39:57.557851+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[18597]: audit 2024-09-10T18:39:57.558696+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[18597]: cluster 2024-09-10T18:39:57.848662+0000 mgr.y (mgr.24512) 241 : cluster [DBG] pgmap v244: 276 pgs: 40 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[31607]: audit 2024-09-10T18:39:57.461189+0000 mon.a (mon.0) 1498 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[31607]: audit 2024-09-10T18:39:57.552353+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[31607]: audit 2024-09-10T18:39:57.552803+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? 172.21.15.100:0/3581556795' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-72601-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[31607]: cluster 2024-09-10T18:39:57.553129+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[31607]: audit 2024-09-10T18:39:57.557851+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[31607]: audit 2024-09-10T18:39:57.558696+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:58 smithi100 bash[31607]: cluster 2024-09-10T18:39:57.848662+0000 mgr.y (mgr.24512) 241 : cluster [DBG] pgmap v244: 276 pgs: 40 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-09-10T18:39:58.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:58 smithi151 bash[22565]: audit 2024-09-10T18:39:57.461189+0000 mon.a (mon.0) 1498 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:58.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:58 smithi151 bash[22565]: audit 2024-09-10T18:39:57.552353+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-72746-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:39:58.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:58 smithi151 bash[22565]: audit 2024-09-10T18:39:57.552803+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? 172.21.15.100:0/3581556795' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-72601-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:39:58.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:58 smithi151 bash[22565]: cluster 2024-09-10T18:39:57.553129+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:39:58.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:58 smithi151 bash[22565]: audit 2024-09-10T18:39:57.557851+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:58.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:58 smithi151 bash[22565]: audit 2024-09-10T18:39:57.558696+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:58.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:58 smithi151 bash[22565]: cluster 2024-09-10T18:39:57.848662+0000 mgr.y (mgr.24512) 241 : cluster [DBG] pgmap v244: 276 pgs: 40 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s wr, 1 op/s 2024-09-10T18:39:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[18597]: audit 2024-09-10T18:39:58.463583+0000 mon.a (mon.0) 1503 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[18597]: audit 2024-09-10T18:39:58.551575+0000 mon.b (mon.1) 111 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[18597]: audit 2024-09-10T18:39:58.562104+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:39:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[18597]: cluster 2024-09-10T18:39:58.562307+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:39:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[18597]: audit 2024-09-10T18:39:58.564439+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[18597]: audit 2024-09-10T18:39:58.567365+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[18597]: audit 2024-09-10T18:39:58.569959+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[31607]: audit 2024-09-10T18:39:58.463583+0000 mon.a (mon.0) 1503 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[31607]: audit 2024-09-10T18:39:58.551575+0000 mon.b (mon.1) 111 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:59.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[31607]: audit 2024-09-10T18:39:58.562104+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:39:59.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[31607]: cluster 2024-09-10T18:39:58.562307+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:39:59.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[31607]: audit 2024-09-10T18:39:58.564439+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:59.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[31607]: audit 2024-09-10T18:39:58.567365+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:59.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:39:59 smithi100 bash[31607]: audit 2024-09-10T18:39:58.569959+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:59.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:59 smithi151 bash[22565]: audit 2024-09-10T18:39:58.463583+0000 mon.a (mon.0) 1503 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:39:59.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:59 smithi151 bash[22565]: audit 2024-09-10T18:39:58.551575+0000 mon.b (mon.1) 111 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:39:59.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:59 smithi151 bash[22565]: audit 2024-09-10T18:39:58.562104+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:39:59.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:59 smithi151 bash[22565]: cluster 2024-09-10T18:39:58.562307+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:39:59.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:59 smithi151 bash[22565]: audit 2024-09-10T18:39:58.564439+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.100:0/1339027163' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:59.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:59 smithi151 bash[22565]: audit 2024-09-10T18:39:58.567365+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]: dispatch 2024-09-10T18:39:59.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:39:59 smithi151 bash[22565]: audit 2024-09-10T18:39:58.569959+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.466086+0000 mon.a (mon.0) 1508 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.566961+0000 mon.a (mon.0) 1509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.567092+0000 mon.a (mon.0) 1510 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:39:59.567427+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.570562+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.572345+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.100:0/2400234120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.574724+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.575224+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.608235+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.608594+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.609168+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.609474+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.609952+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:39:59.610255+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:39:59.849602+0000 mgr.y (mgr.24512) 242 : cluster [DBG] pgmap v247: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.466086+0000 mon.a (mon.0) 1508 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.566961+0000 mon.a (mon.0) 1509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:40:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.567092+0000 mon.a (mon.0) 1510 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:39:59.567427+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.570562+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.572345+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.100:0/2400234120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.574724+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.575224+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.608235+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.608594+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.609168+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.609474+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.609952+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:39:59.610255+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:39:59.849602+0000 mgr.y (mgr.24512) 242 : cluster [DBG] pgmap v247: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.000184+0000 mon.a (mon.0) 1517 : cluster [WRN] Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.000231+0000 mon.a (mon.0) 1518 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.000252+0000 mon.a (mon.0) 1519 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.000267+0000 mon.a (mon.0) 1520 : cluster [WRN] application not enabled on pool 'LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20' 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.000286+0000 mon.a (mon.0) 1521 : cluster [WRN] application not enabled on pool 'LibRadosMiscECPP_smithi100-72746-36' 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.000338+0000 mon.a (mon.0) 1522 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.291880+0000 mon.a (mon.0) 1523 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.303690+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.304048+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.304899+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:00.305071+0000 mon.a (mon.0) 1527 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.308336+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.308681+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.310128+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.310583+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.314028+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.000184+0000 mon.a (mon.0) 1517 : cluster [WRN] Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.000231+0000 mon.a (mon.0) 1518 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.000252+0000 mon.a (mon.0) 1519 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.000267+0000 mon.a (mon.0) 1520 : cluster [WRN] application not enabled on pool 'LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20' 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.000286+0000 mon.a (mon.0) 1521 : cluster [WRN] application not enabled on pool 'LibRadosMiscECPP_smithi100-72746-36' 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.000338+0000 mon.a (mon.0) 1522 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:40:00.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.291880+0000 mon.a (mon.0) 1523 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.303690+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.304048+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.304899+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:00.305071+0000 mon.a (mon.0) 1527 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.308336+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.308681+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.310128+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.310583+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.314028+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.324530+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:00.799 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.325900+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.800 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[18597]: audit 2024-09-10T18:40:00.340905+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:00.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.324530+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:00.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.325900+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.800 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:00 smithi100 bash[31607]: audit 2024-09-10T18:40:00.340905+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:00.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.466086+0000 mon.a (mon.0) 1508 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:00.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.566961+0000 mon.a (mon.0) 1509 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-72593-27"}]': finished 2024-09-10T18:40:00.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.567092+0000 mon.a (mon.0) 1510 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:00.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:39:59.567427+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:40:00.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.570562+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.572345+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.100:0/2400234120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:00.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.574724+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.575224+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:00.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.608235+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.608594+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.609168+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.609474+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.609952+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:39:59.610255+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:39:59.849602+0000 mgr.y (mgr.24512) 242 : cluster [DBG] pgmap v247: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.000184+0000 mon.a (mon.0) 1517 : cluster [WRN] Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.000231+0000 mon.a (mon.0) 1518 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.000252+0000 mon.a (mon.0) 1519 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.000267+0000 mon.a (mon.0) 1520 : cluster [WRN] application not enabled on pool 'LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20' 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.000286+0000 mon.a (mon.0) 1521 : cluster [WRN] application not enabled on pool 'LibRadosMiscECPP_smithi100-72746-36' 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.000338+0000 mon.a (mon.0) 1522 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.291880+0000 mon.a (mon.0) 1523 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.303690+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.304048+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-72601-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.304899+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-72593-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:00.305071+0000 mon.a (mon.0) 1527 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.308336+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.100:0/2504244726' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.308681+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.310128+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]: dispatch 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.310583+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.314028+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.324530+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.325900+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:00 smithi151 bash[22565]: audit 2024-09-10T18:40:00.340905+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForCompl9662 ms total) 2024-09-10T18:40:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-09-10T18:40:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-10T18:40:01.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (168 ms) 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (168 ms total) 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (7 ms) 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (5 ms) 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (12 ms total) 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-10T18:40:01.318 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (30 ms) 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (4 ms) 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (34 ms total) 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (35 ms) 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (6 ms) 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (41 ms total) 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-10T18:40:01.319 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1036 ms) 2024-09-10T18:40:01.320 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1036 ms total) 2024-09-10T18:40:01.320 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-09-10T18:40:01.320 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-10T18:40:01.320 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (106934 ms total) 2024-09-10T18:40:01.320 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-10T18:40:01.741 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:40:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:01] "GET /metrics HTTP/1.1" 200 238141 "" "Prometheus/2.43.0" 2024-09-10T18:40:02.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:01 smithi100 bash[18597]: audit 2024-09-10T18:40:00.470755+0000 mon.a (mon.0) 1532 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:02.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:01 smithi100 bash[18597]: audit 2024-09-10T18:40:01.308124+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:40:02.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:01 smithi100 bash[31607]: audit 2024-09-10T18:40:00.470755+0000 mon.a (mon.0) 1532 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:02.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:01 smithi100 bash[31607]: audit 2024-09-10T18:40:01.308124+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:40:02.155 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:01 smithi151 bash[22565]: audit 2024-09-10T18:40:00.470755+0000 mon.a (mon.0) 1532 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:02.155 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:01 smithi151 bash[22565]: audit 2024-09-10T18:40:01.308124+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-72746-36"}]': finished 2024-09-10T18:40:02.323 INFO:tasks.workunit.client.0.smithi100.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-10T18:40:02.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-10T18:40:02.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4062 ms) 2024-09-10T18:40:02.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-10T18:40:02.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3978 ms) 2024-09-10T18:40:02.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-10T18:40:02.323 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4023 ms) 2024-09-10T18:40:02.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12063 ms total) 2024-09-10T18:40:02.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-09-10T18:40:02.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-10T18:40:02.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (107921 ms total) 2024-09-10T18:40:02.324 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-10T18:40:02.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.308260+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.313095+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.716367+0000 mon.b (mon.1) 113 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.730767+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: cluster 2024-09-10T18:40:01.731105+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.733601+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.742753+0000 mon.a (mon.0) 1538 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.743156+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: cluster 2024-09-10T18:40:01.850356+0000 mgr.y (mgr.24512) 243 : cluster [DBG] pgmap v250: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:03.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:01.851006+0000 mon.c (mon.2) 261 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.157542+0000 mon.a (mon.0) 1540 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.305156+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]: dispatch 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.315743+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.316101+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.308260+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.313095+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.716367+0000 mon.b (mon.1) 113 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.730767+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: cluster 2024-09-10T18:40:01.731105+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:40:03.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.733601+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.742753+0000 mon.a (mon.0) 1538 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.743156+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: cluster 2024-09-10T18:40:01.850356+0000 mgr.y (mgr.24512) 243 : cluster [DBG] pgmap v250: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:01.851006+0000 mon.c (mon.2) 261 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.157542+0000 mon.a (mon.0) 1540 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.305156+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]: dispatch 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.315743+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.316101+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.316300+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]': finished 2024-09-10T18:40:03.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.316614+0000 mon.a (mon.0) 1544 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: cluster 2024-09-10T18:40:02.316785+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.322018+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]: dispatch 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[18597]: audit 2024-09-10T18:40:02.324495+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.100:0/2170116871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-72601-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.316300+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]': finished 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.316614+0000 mon.a (mon.0) 1544 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: cluster 2024-09-10T18:40:02.316785+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.322018+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]: dispatch 2024-09-10T18:40:03.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:02 smithi100 bash[31607]: audit 2024-09-10T18:40:02.324495+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.100:0/2170116871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-72601-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:03.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.308260+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:40:03.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.313095+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.100:0/379470274' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:03.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.716367+0000 mon.b (mon.1) 113 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:03.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.730767+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:03.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: cluster 2024-09-10T18:40:01.731105+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.733601+0000 mon.a (mon.0) 1537 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]: dispatch 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.742753+0000 mon.a (mon.0) 1538 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.743156+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: cluster 2024-09-10T18:40:01.850356+0000 mgr.y (mgr.24512) 243 : cluster [DBG] pgmap v250: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:01.851006+0000 mon.c (mon.2) 261 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.157542+0000 mon.a (mon.0) 1540 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.305156+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]: dispatch 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.315743+0000 mon.a (mon.0) 1541 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-72593-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.316101+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-72806-20"}]': finished 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.316300+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-15"}]': finished 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.316614+0000 mon.a (mon.0) 1544 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: cluster 2024-09-10T18:40:02.316785+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:40:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.322018+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]: dispatch 2024-09-10T18:40:03.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:02 smithi151 bash[22565]: audit 2024-09-10T18:40:02.324495+0000 mon.a (mon.0) 1547 : audit [INF] from='client.? 172.21.15.100:0/2170116871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-72601-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:04.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[18597]: audit 2024-09-10T18:40:02.155018+0000 mgr.y (mgr.24512) 244 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:04.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[18597]: audit 2024-09-10T18:40:02.746771+0000 mon.a (mon.0) 1548 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:04.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[18597]: cluster 2024-09-10T18:40:03.305698+0000 mon.a (mon.0) 1549 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:04.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[18597]: audit 2024-09-10T18:40:03.316771+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]': finished 2024-09-10T18:40:04.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[18597]: audit 2024-09-10T18:40:03.317167+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? 172.21.15.100:0/2170116871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-72601-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:04.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[18597]: cluster 2024-09-10T18:40:03.317246+0000 mon.a (mon.0) 1552 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:40:04.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[31607]: audit 2024-09-10T18:40:02.155018+0000 mgr.y (mgr.24512) 244 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:04.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[31607]: audit 2024-09-10T18:40:02.746771+0000 mon.a (mon.0) 1548 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:04.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[31607]: cluster 2024-09-10T18:40:03.305698+0000 mon.a (mon.0) 1549 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:04.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[31607]: audit 2024-09-10T18:40:03.316771+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]': finished 2024-09-10T18:40:04.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[31607]: audit 2024-09-10T18:40:03.317167+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? 172.21.15.100:0/2170116871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-72601-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:04.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:03 smithi100 bash[31607]: cluster 2024-09-10T18:40:03.317246+0000 mon.a (mon.0) 1552 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:40:04.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:03 smithi151 bash[22565]: audit 2024-09-10T18:40:02.155018+0000 mgr.y (mgr.24512) 244 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:04.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:03 smithi151 bash[22565]: audit 2024-09-10T18:40:02.746771+0000 mon.a (mon.0) 1548 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:04.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:03 smithi151 bash[22565]: cluster 2024-09-10T18:40:03.305698+0000 mon.a (mon.0) 1549 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:04.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:03 smithi151 bash[22565]: audit 2024-09-10T18:40:03.316771+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-15", "mode": "writeback"}]': finished 2024-09-10T18:40:04.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:03 smithi151 bash[22565]: audit 2024-09-10T18:40:03.317167+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? 172.21.15.100:0/2170116871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-72601-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:04.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:03 smithi151 bash[22565]: cluster 2024-09-10T18:40:03.317246+0000 mon.a (mon.0) 1552 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:40:04.928 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:04.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:03.748758+0000 mon.a (mon.0) 1553 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:04.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:03.749652+0000 mon.a (mon.0) 1554 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:04.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:03.750517+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:40:04.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: cluster 2024-09-10T18:40:03.851347+0000 mgr.y (mgr.24512) 245 : cluster [DBG] pgmap v253: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:04.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:04.321769+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:40:04.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: cluster 2024-09-10T18:40:04.321860+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:40:04.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:04.322908+0000 mon.a (mon.0) 1558 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:04.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:04.324042+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:04.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:04.324875+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:04.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:04 smithi151 bash[22565]: audit 2024-09-10T18:40:04.353432+0000 mon.b (mon.1) 115 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:03.748758+0000 mon.a (mon.0) 1553 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:03.749652+0000 mon.a (mon.0) 1554 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:03.750517+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:40:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: cluster 2024-09-10T18:40:03.851347+0000 mgr.y (mgr.24512) 245 : cluster [DBG] pgmap v253: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:04.321769+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: cluster 2024-09-10T18:40:04.321860+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:04.322908+0000 mon.a (mon.0) 1558 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:04.324042+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:04.324875+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[18597]: audit 2024-09-10T18:40:04.353432+0000 mon.b (mon.1) 115 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:03.748758+0000 mon.a (mon.0) 1553 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:03.749652+0000 mon.a (mon.0) 1554 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:03.750517+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:40:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: cluster 2024-09-10T18:40:03.851347+0000 mgr.y (mgr.24512) 245 : cluster [DBG] pgmap v253: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:04.321769+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:40:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: cluster 2024-09-10T18:40:04.321860+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:40:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:04.322908+0000 mon.a (mon.0) 1558 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:04.324042+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:04.324875+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:04 smithi100 bash[31607]: audit 2024-09-10T18:40:04.353432+0000 mon.b (mon.1) 115 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:06.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:04.786013+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:06.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:04.786538+0000 mon.a (mon.0) 1561 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:04.794379+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: cluster 2024-09-10T18:40:05.295470+0000 mon.a (mon.0) 1563 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.311681+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.324945+0000 mon.a (mon.0) 1564 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.325146+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.325258+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: cluster 2024-09-10T18:40:05.325447+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.326782+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.327382+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.100:0/260767972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.328622+0000 mon.a (mon.0) 1568 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.329371+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.329848+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:05 smithi151 bash[22565]: audit 2024-09-10T18:40:05.330180+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:06.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:04.786013+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:04.786538+0000 mon.a (mon.0) 1561 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:04.794379+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: cluster 2024-09-10T18:40:05.295470+0000 mon.a (mon.0) 1563 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.311681+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.324945+0000 mon.a (mon.0) 1564 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.325146+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.325258+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: cluster 2024-09-10T18:40:05.325447+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.326782+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.327382+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.100:0/260767972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.328622+0000 mon.a (mon.0) 1568 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.329371+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.329848+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[18597]: audit 2024-09-10T18:40:05.330180+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:04.786013+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:04.786538+0000 mon.a (mon.0) 1561 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:04.794379+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: cluster 2024-09-10T18:40:05.295470+0000 mon.a (mon.0) 1563 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.311681+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.324945+0000 mon.a (mon.0) 1564 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.325146+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.325258+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: cluster 2024-09-10T18:40:05.325447+0000 mon.a (mon.0) 1567 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.326782+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.100:0/2119341470' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.327382+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.100:0/260767972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.328622+0000 mon.a (mon.0) 1568 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.329371+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]: dispatch 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.329848+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]: dispatch 2024-09-10T18:40:06.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:05 smithi100 bash[31607]: audit 2024-09-10T18:40:05.330180+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:07.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: cluster 2024-09-10T18:40:05.852305+0000 mgr.y (mgr.24512) 246 : cluster [DBG] pgmap v256: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:05.852959+0000 mon.c (mon.2) 265 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:05.855257+0000 mon.a (mon.0) 1572 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: cluster 2024-09-10T18:40:06.316773+0000 mon.a (mon.0) 1573 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.329333+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.329640+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]': finished 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.330181+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.330543+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: cluster 2024-09-10T18:40:06.330632+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.331519+0000 mon.a (mon.0) 1579 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.362966+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.363299+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.364177+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.364511+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.365114+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:07.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:06 smithi151 bash[22565]: audit 2024-09-10T18:40:06.365441+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: cluster 2024-09-10T18:40:05.852305+0000 mgr.y (mgr.24512) 246 : cluster [DBG] pgmap v256: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:05.852959+0000 mon.c (mon.2) 265 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:05.855257+0000 mon.a (mon.0) 1572 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: cluster 2024-09-10T18:40:06.316773+0000 mon.a (mon.0) 1573 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.329333+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.329640+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]': finished 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.330181+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.330543+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: cluster 2024-09-10T18:40:06.330632+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:40:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.331519+0000 mon.a (mon.0) 1579 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.362966+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.363299+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.364177+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.364511+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.365114+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[18597]: audit 2024-09-10T18:40:06.365441+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: cluster 2024-09-10T18:40:05.852305+0000 mgr.y (mgr.24512) 246 : cluster [DBG] pgmap v256: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:05.852959+0000 mon.c (mon.2) 265 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:05.855257+0000 mon.a (mon.0) 1572 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: cluster 2024-09-10T18:40:06.316773+0000 mon.a (mon.0) 1573 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.329333+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-72593-28"}]': finished 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.329640+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-15"}]': finished 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.330181+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-72601-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.330543+0000 mon.a (mon.0) 1577 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: cluster 2024-09-10T18:40:06.330632+0000 mon.a (mon.0) 1578 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.331519+0000 mon.a (mon.0) 1579 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.362966+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.363299+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.364177+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.364511+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.365114+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:06 smithi100 bash[31607]: audit 2024-09-10T18:40:06.365441+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:08.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: audit 2024-09-10T18:40:07.330780+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:08.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: cluster 2024-09-10T18:40:07.331279+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:40:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: audit 2024-09-10T18:40:07.332446+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: audit 2024-09-10T18:40:07.336321+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: audit 2024-09-10T18:40:07.336971+0000 mon.a (mon.0) 1586 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: cluster 2024-09-10T18:40:07.853244+0000 mgr.y (mgr.24512) 247 : cluster [DBG] pgmap v259: 228 pgs: 228 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:40:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: audit 2024-09-10T18:40:07.854014+0000 mon.c (mon.2) 270 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:40:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:08 smithi151 bash[22565]: audit 2024-09-10T18:40:08.173507+0000 mon.a (mon.0) 1587 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:40:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: audit 2024-09-10T18:40:07.330780+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: cluster 2024-09-10T18:40:07.331279+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:40:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: audit 2024-09-10T18:40:07.332446+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: audit 2024-09-10T18:40:07.336321+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:08.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: audit 2024-09-10T18:40:07.336971+0000 mon.a (mon.0) 1586 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:08.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: cluster 2024-09-10T18:40:07.853244+0000 mgr.y (mgr.24512) 247 : cluster [DBG] pgmap v259: 228 pgs: 228 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:40:08.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: audit 2024-09-10T18:40:07.854014+0000 mon.c (mon.2) 270 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:40:08.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[18597]: audit 2024-09-10T18:40:08.173507+0000 mon.a (mon.0) 1587 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:40:08.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: audit 2024-09-10T18:40:07.330780+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-72593-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:08.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: cluster 2024-09-10T18:40:07.331279+0000 mon.a (mon.0) 1584 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:40:08.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: audit 2024-09-10T18:40:07.332446+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:08.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: audit 2024-09-10T18:40:07.336321+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:08.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: audit 2024-09-10T18:40:07.336971+0000 mon.a (mon.0) 1586 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:08.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: cluster 2024-09-10T18:40:07.853244+0000 mgr.y (mgr.24512) 247 : cluster [DBG] pgmap v259: 228 pgs: 228 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:40:08.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: audit 2024-09-10T18:40:07.854014+0000 mon.c (mon.2) 270 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:40:08.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:08 smithi100 bash[31607]: audit 2024-09-10T18:40:08.173507+0000 mon.a (mon.0) 1587 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:40:09.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.748280+0000 mon.b (mon.1) 117 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.762136+0000 mon.a (mon.0) 1588 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:40:09.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: cluster 2024-09-10T18:40:08.762362+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:40:09.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.763232+0000 mon.a (mon.0) 1590 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.763776+0000 mon.c (mon.2) 271 : audit [INF] from='client.? 172.21.15.100:0/4089061494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.768107+0000 mon.a (mon.0) 1591 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.769938+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.770633+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:40:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:08.771488+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:09 smithi151 bash[22565]: audit 2024-09-10T18:40:09.333052+0000 mon.a (mon.0) 1595 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.748280+0000 mon.b (mon.1) 117 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.762136+0000 mon.a (mon.0) 1588 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:40:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: cluster 2024-09-10T18:40:08.762362+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:40:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.763232+0000 mon.a (mon.0) 1590 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.763776+0000 mon.c (mon.2) 271 : audit [INF] from='client.? 172.21.15.100:0/4089061494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.768107+0000 mon.a (mon.0) 1591 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.769938+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.770633+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:40:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:08.771488+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[18597]: audit 2024-09-10T18:40:09.333052+0000 mon.a (mon.0) 1595 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.748280+0000 mon.b (mon.1) 117 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.762136+0000 mon.a (mon.0) 1588 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-72633-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: cluster 2024-09-10T18:40:08.762362+0000 mon.a (mon.0) 1589 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.763232+0000 mon.a (mon.0) 1590 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.763776+0000 mon.c (mon.2) 271 : audit [INF] from='client.? 172.21.15.100:0/4089061494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.768107+0000 mon.a (mon.0) 1591 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.769938+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.770633+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:08.771488+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:09 smithi100 bash[31607]: audit 2024-09-10T18:40:09.333052+0000 mon.a (mon.0) 1595 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:10.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-10T18:40:10.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-10T18:40:10.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-10T18:40:10.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-10T18:40:10.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-10T18:40:10.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (11506 ms) 2024-09-10T18:40:10.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (26001 ms) 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6595 ms) 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2900 ms) 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3277 ms) 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3425 ms) 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2293 ms) 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-10T18:40:10.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3019 ms) 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2992 ms) 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3015 ms) 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2997 ms) 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3027 ms) 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3050 ms) 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3432 ms) 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-10T18:40:10.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (2626 ms) 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3434 ms) 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2594 ms) 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3015 ms) 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3446 ms) 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (2590 ms) 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3022 ms) 2024-09-10T18:40:10.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3000 ms) 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3019 ms) 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3164 ms) 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2593 ms) 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3008 ms) 2024-09-10T18:40:10.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-10T18:40:11.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.335688+0000 mon.c (mon.2) 272 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.346161+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.346440+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.346555+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.346657+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: cluster 2024-09-10T18:40:09.346755+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.349133+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.372759+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:09.390057+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:11.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: cluster 2024-09-10T18:40:09.854271+0000 mgr.y (mgr.24512) 248 : cluster [DBG] pgmap v262: 299 pgs: 72 unknown, 227 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:10.294476+0000 mon.b (mon.1) 119 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: cluster 2024-09-10T18:40:10.297211+0000 mon.a (mon.0) 1603 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:10.308783+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: cluster 2024-09-10T18:40:10.309074+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:10.312548+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[18597]: audit 2024-09-10T18:40:10.340571+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.335688+0000 mon.c (mon.2) 272 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:11.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.346161+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.346440+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.346555+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.346657+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: cluster 2024-09-10T18:40:09.346755+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.349133+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.372759+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:09.390057+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: cluster 2024-09-10T18:40:09.854271+0000 mgr.y (mgr.24512) 248 : cluster [DBG] pgmap v262: 299 pgs: 72 unknown, 227 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:10.294476+0000 mon.b (mon.1) 119 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: cluster 2024-09-10T18:40:10.297211+0000 mon.a (mon.0) 1603 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:11.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:10.308783+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:11.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: cluster 2024-09-10T18:40:10.309074+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:40:11.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:10.312548+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:11.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:10 smithi100 bash[31607]: audit 2024-09-10T18:40:10.340571+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.335688+0000 mon.c (mon.2) 272 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:11.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.346161+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-72593-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.346440+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-72601-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.346555+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-72633-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.346657+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: cluster 2024-09-10T18:40:09.346755+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.349133+0000 mon.a (mon.0) 1601 : audit [DBG] from='client.? 172.21.15.100:0/4233284596' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.372759+0000 mon.b (mon.1) 118 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:09.390057+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: cluster 2024-09-10T18:40:09.854271+0000 mgr.y (mgr.24512) 248 : cluster [DBG] pgmap v262: 299 pgs: 72 unknown, 227 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:10.294476+0000 mon.b (mon.1) 119 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: cluster 2024-09-10T18:40:10.297211+0000 mon.a (mon.0) 1603 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:10.308783+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: cluster 2024-09-10T18:40:10.309074+0000 mon.a (mon.0) 1605 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:40:11.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:10.312548+0000 mon.a (mon.0) 1606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:11.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:10 smithi151 bash[22565]: audit 2024-09-10T18:40:10.340571+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:11.777 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:11] "GET /metrics HTTP/1.1" 200 234742 "" "Prometheus/2.43.0" 2024-09-10T18:40:12.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:10.764333+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:10.765053+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-72633-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.300403+0000 mon.b (mon.1) 120 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.314814+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]': finished 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.315180+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-72633-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: cluster 2024-09-10T18:40:11.315521+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.318460+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-72633-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.319789+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.319845+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.320795+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.100:0/1494495618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.320799+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:12.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[18597]: audit 2024-09-10T18:40:11.321489+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:10.764333+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:10.765053+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-72633-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.300403+0000 mon.b (mon.1) 120 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]: dispatch 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.314814+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]': finished 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.315180+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-72633-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: cluster 2024-09-10T18:40:11.315521+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.318460+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-72633-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:12.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.319789+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]: dispatch 2024-09-10T18:40:12.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.319845+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:12.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.320795+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.100:0/1494495618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:12.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.320799+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:12.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:11 smithi100 bash[31607]: audit 2024-09-10T18:40:11.321489+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:10.764333+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:10.765053+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-72633-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.300403+0000 mon.b (mon.1) 120 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]: dispatch 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.314814+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-17"}]': finished 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.315180+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-72633-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: cluster 2024-09-10T18:40:11.315521+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.318460+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-72633-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.319789+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]: dispatch 2024-09-10T18:40:12.167 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.319845+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:12.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.320795+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.100:0/1494495618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:12.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.320799+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:12.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:11 smithi151 bash[22565]: audit 2024-09-10T18:40:11.321489+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:12.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:40:13.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: cluster 2024-09-10T18:40:11.855146+0000 mgr.y (mgr.24512) 249 : cluster [DBG] pgmap v265: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:13.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: cluster 2024-09-10T18:40:12.306236+0000 mon.a (mon.0) 1617 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:13.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: audit 2024-09-10T18:40:12.742423+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]': finished 2024-09-10T18:40:13.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: audit 2024-09-10T18:40:12.742809+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: audit 2024-09-10T18:40:12.743015+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: cluster 2024-09-10T18:40:12.743439+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: audit 2024-09-10T18:40:12.743978+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[31607]: audit 2024-09-10T18:40:12.749722+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: cluster 2024-09-10T18:40:11.855146+0000 mgr.y (mgr.24512) 249 : cluster [DBG] pgmap v265: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: cluster 2024-09-10T18:40:12.306236+0000 mon.a (mon.0) 1617 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: audit 2024-09-10T18:40:12.742423+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]': finished 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: audit 2024-09-10T18:40:12.742809+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: audit 2024-09-10T18:40:12.743015+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: cluster 2024-09-10T18:40:12.743439+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:40:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: audit 2024-09-10T18:40:12.743978+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:13.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:12 smithi100 bash[18597]: audit 2024-09-10T18:40:12.749722+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:13.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: cluster 2024-09-10T18:40:11.855146+0000 mgr.y (mgr.24512) 249 : cluster [DBG] pgmap v265: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:13.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: cluster 2024-09-10T18:40:12.306236+0000 mon.a (mon.0) 1617 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:13.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: audit 2024-09-10T18:40:12.742423+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-17", "mode": "writeback"}]': finished 2024-09-10T18:40:13.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: audit 2024-09-10T18:40:12.742809+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:13.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: audit 2024-09-10T18:40:12.743015+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-72601-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:13.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: cluster 2024-09-10T18:40:12.743439+0000 mon.a (mon.0) 1621 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:40:13.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: audit 2024-09-10T18:40:12.743978+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.100:0/1944865520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:13.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:12 smithi151 bash[22565]: audit 2024-09-10T18:40:12.749722+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]: dispatch 2024-09-10T18:40:14.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:12.166594+0000 mgr.y (mgr.24512) 250 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:12.801225+0000 mon.b (mon.1) 121 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:12.818099+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.307287+0000 mon.b (mon.1) 122 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.320885+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-72633-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.321108+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.321249+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: cluster 2024-09-10T18:40:13.321341+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.324419+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.756536+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.758888+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[31607]: audit 2024-09-10T18:40:13.759433+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-72593-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:12.166594+0000 mgr.y (mgr.24512) 250 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:12.801225+0000 mon.b (mon.1) 121 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:12.818099+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.307287+0000 mon.b (mon.1) 122 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.320885+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-72633-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.321108+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.321249+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:14.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: cluster 2024-09-10T18:40:13.321341+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:40:14.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.324419+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:14.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.756536+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:14.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.758888+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:14.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:13 smithi100 bash[18597]: audit 2024-09-10T18:40:13.759433+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-72593-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:14.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:12.166594+0000 mgr.y (mgr.24512) 250 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:14.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:12.801225+0000 mon.b (mon.1) 121 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:14.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:12.818099+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:14.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.307287+0000 mon.b (mon.1) 122 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.320885+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-72633-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.321108+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-72593-29"}]': finished 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.321249+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: cluster 2024-09-10T18:40:13.321341+0000 mon.a (mon.0) 1627 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.324419+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]: dispatch 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.756536+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.758888+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:14.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:13 smithi151 bash[22565]: audit 2024-09-10T18:40:13.759433+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-72593-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAixpected=1 2024-09-10T18:40:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:863748b0:::15:head 2024-09-10T18:40:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:863748b0:::15:head expected=23:863748b0:::15:head 2024-09-10T18:40:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:863748b0:::15:head -> 15 2024-09-10T18:40:14.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=15 expected=15 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:6cac518f:::0:head 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:6cac518f:::0:head expected=23:6cac518f:::0:head 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:6cac518f:::0:head -> 0 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=0 expected=0 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:62a1935d:::14:head 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:62a1935d:::14:head expected=23:62a1935d:::14:head 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:62a1935d:::14:head -> 14 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=14 expected=14 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:566253c9:::13:head 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:566253c9:::13:head expected=23:566253c9:::13:head 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:566253c9:::13:head -> 13 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=13 expected=13 2024-09-10T18:40:14.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:c4fdafeb:::6:head 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:c4fdafeb:::6:head expected=23:c4fdafeb:::6:head 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:c4fdafeb:::6:head -> 6 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=6 expected=6 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:bd63b0f1:::8:head 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:bd63b0f1:::8:head expected=23:bd63b0f1:::8:head 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:bd63b0f1:::8:head -> 8 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=8 expected=8 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:b29083e3:::5:head 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:b29083e3:::5:head expected=23:b29083e3:::5:head 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:b29083e3:::5:head -> 5 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=5 expected=5 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 23:89d3ae78:::11:head 2024-09-10T18:40:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=23:89d3ae78:::11:head expected=23:89d3ae78:::11:head 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 23:89d3ae78:::11:head -> 11 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=11 expected=11 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (419 ms) 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (97388 ms) 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 0/5 -> MIN 23:33333333::::head 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 1/5 -> 23:33333333::::head 23:66666666::::head 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 2/5 -> 23:66666666::::head 23:99999999::::head 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 3/5 -> 23:99999999::::head 23:cccccccc::::head 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 4/5 -> 23:cccccccc::::head MAX 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (5027 ms) 2024-09-10T18:40:14.405 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 7 tests from LibRadosList (103136 ms total) 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1033 ms) 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo2,foo3 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo2 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo3 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo4,foo5 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo4 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo5 2024-09-10T18:40:14.406 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo6,foo7 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo7 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo6 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo4 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo5 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo7 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo6 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo1 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo1 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo2 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo3 2024-09-10T18:40:14.407 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (47 ms) 2024-09-10T18:40:14.928 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: cluster 2024-09-10T18:40:13.856002+0000 mgr.y (mgr.24512) 251 : cluster [DBG] pgmap v268: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: cluster 2024-09-10T18:40:14.313689+0000 mon.a (mon.0) 1632 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: audit 2024-09-10T18:40:14.326604+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]': finished 2024-09-10T18:40:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: audit 2024-09-10T18:40:14.326932+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-72593-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:14.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: cluster 2024-09-10T18:40:14.327024+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:40:14.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: audit 2024-09-10T18:40:14.329634+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.100:0/641205704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:14.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: audit 2024-09-10T18:40:14.330874+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-72593-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:14.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:14 smithi151 bash[22565]: audit 2024-09-10T18:40:14.332179+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:15.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: cluster 2024-09-10T18:40:13.856002+0000 mgr.y (mgr.24512) 251 : cluster [DBG] pgmap v268: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:15.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: cluster 2024-09-10T18:40:14.313689+0000 mon.a (mon.0) 1632 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:15.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: audit 2024-09-10T18:40:14.326604+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]': finished 2024-09-10T18:40:15.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: audit 2024-09-10T18:40:14.326932+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-72593-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: cluster 2024-09-10T18:40:14.327024+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: audit 2024-09-10T18:40:14.329634+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.100:0/641205704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: audit 2024-09-10T18:40:14.330874+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-72593-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[31607]: audit 2024-09-10T18:40:14.332179+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: cluster 2024-09-10T18:40:13.856002+0000 mgr.y (mgr.24512) 251 : cluster [DBG] pgmap v268: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: cluster 2024-09-10T18:40:14.313689+0000 mon.a (mon.0) 1632 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: audit 2024-09-10T18:40:14.326604+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-17"}]': finished 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: audit 2024-09-10T18:40:14.326932+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-72593-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: cluster 2024-09-10T18:40:14.327024+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:40:15.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: audit 2024-09-10T18:40:14.329634+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.100:0/641205704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:15.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: audit 2024-09-10T18:40:14.330874+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-72593-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:15.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:14 smithi100 bash[18597]: audit 2024-09-10T18:40:14.332179+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:16.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[18597]: cluster 2024-09-10T18:40:15.300572+0000 mon.a (mon.0) 1638 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:16.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[18597]: audit 2024-09-10T18:40:15.766513+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:16.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[18597]: cluster 2024-09-10T18:40:15.767042+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:40:16.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[18597]: audit 2024-09-10T18:40:15.769636+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:16.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[31607]: cluster 2024-09-10T18:40:15.300572+0000 mon.a (mon.0) 1638 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[31607]: audit 2024-09-10T18:40:15.766513+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[31607]: cluster 2024-09-10T18:40:15.767042+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:40:16.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:16 smithi100 bash[31607]: audit 2024-09-10T18:40:15.769636+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:16.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:16 smithi151 bash[22565]: cluster 2024-09-10T18:40:15.300572+0000 mon.a (mon.0) 1638 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:16.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:16 smithi151 bash[22565]: audit 2024-09-10T18:40:15.766513+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-72601-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:16.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:16 smithi151 bash[22565]: cluster 2024-09-10T18:40:15.767042+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:40:16.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:16 smithi151 bash[22565]: audit 2024-09-10T18:40:15.769636+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[18597]: cluster 2024-09-10T18:40:15.856783+0000 mgr.y (mgr.24512) 252 : cluster [DBG] pgmap v271: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[18597]: audit 2024-09-10T18:40:16.335838+0000 mon.b (mon.1) 123 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[18597]: audit 2024-09-10T18:40:16.341203+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-72593-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[18597]: audit 2024-09-10T18:40:16.341545+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[18597]: cluster 2024-09-10T18:40:16.341825+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:40:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[18597]: audit 2024-09-10T18:40:16.346409+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[18597]: audit 2024-09-10T18:40:16.352605+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[31607]: cluster 2024-09-10T18:40:15.856783+0000 mgr.y (mgr.24512) 252 : cluster [DBG] pgmap v271: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[31607]: audit 2024-09-10T18:40:16.335838+0000 mon.b (mon.1) 123 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[31607]: audit 2024-09-10T18:40:16.341203+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-72593-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[31607]: audit 2024-09-10T18:40:16.341545+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[31607]: cluster 2024-09-10T18:40:16.341825+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:40:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[31607]: audit 2024-09-10T18:40:16.346409+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:17.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:17 smithi100 bash[31607]: audit 2024-09-10T18:40:16.352605+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:17 smithi151 bash[22565]: cluster 2024-09-10T18:40:15.856783+0000 mgr.y (mgr.24512) 252 : cluster [DBG] pgmap v271: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:17 smithi151 bash[22565]: audit 2024-09-10T18:40:16.335838+0000 mon.b (mon.1) 123 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:17 smithi151 bash[22565]: audit 2024-09-10T18:40:16.341203+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-72593-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:17.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:17 smithi151 bash[22565]: audit 2024-09-10T18:40:16.341545+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:17.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:17 smithi151 bash[22565]: cluster 2024-09-10T18:40:16.341825+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:40:17.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:17 smithi151 bash[22565]: audit 2024-09-10T18:40:16.346409+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]: dispatch 2024-09-10T18:40:17.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:17 smithi151 bash[22565]: audit 2024-09-10T18:40:16.352605+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:18.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: audit 2024-09-10T18:40:17.338181+0000 mon.b (mon.1) 124 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:18.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: audit 2024-09-10T18:40:17.344201+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:18.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: audit 2024-09-10T18:40:17.344445+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:18.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: cluster 2024-09-10T18:40:17.344628+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:40:18.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: audit 2024-09-10T18:40:17.347144+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.100:0/3820055533' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:18.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: audit 2024-09-10T18:40:17.347779+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:18.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: audit 2024-09-10T18:40:17.354897+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:18.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:18 smithi151 bash[22565]: cluster 2024-09-10T18:40:17.857645+0000 mgr.y (mgr.24512) 253 : cluster [DBG] pgmap v274: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:18.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: audit 2024-09-10T18:40:17.338181+0000 mon.b (mon.1) 124 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: audit 2024-09-10T18:40:17.344201+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: audit 2024-09-10T18:40:17.344445+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: cluster 2024-09-10T18:40:17.344628+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: audit 2024-09-10T18:40:17.347144+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.100:0/3820055533' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: audit 2024-09-10T18:40:17.347779+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: audit 2024-09-10T18:40:17.354897+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[31607]: cluster 2024-09-10T18:40:17.857645+0000 mgr.y (mgr.24512) 253 : cluster [DBG] pgmap v274: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: audit 2024-09-10T18:40:17.338181+0000 mon.b (mon.1) 124 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: audit 2024-09-10T18:40:17.344201+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.100:0/220513018' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-72633-2"}]': finished 2024-09-10T18:40:18.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: audit 2024-09-10T18:40:17.344445+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:18.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: cluster 2024-09-10T18:40:17.344628+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:40:18.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: audit 2024-09-10T18:40:17.347144+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.100:0/3820055533' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:18.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: audit 2024-09-10T18:40:17.347779+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:18.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: audit 2024-09-10T18:40:17.354897+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:18.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:18 smithi100 bash[18597]: cluster 2024-09-10T18:40:17.857645+0000 mgr.y (mgr.24512) 253 : cluster [DBG] pgmap v274: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:19.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:19 smithi151 bash[22565]: audit 2024-09-10T18:40:18.346779+0000 mon.b (mon.1) 125 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:19.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:19 smithi151 bash[22565]: audit 2024-09-10T18:40:18.359276+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:19.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:19 smithi151 bash[22565]: audit 2024-09-10T18:40:18.359455+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:19.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:19 smithi151 bash[22565]: cluster 2024-09-10T18:40:18.359719+0000 mon.a (mon.0) 1654 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:40:19.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:19 smithi151 bash[22565]: audit 2024-09-10T18:40:18.363826+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:19.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:19 smithi151 bash[22565]: audit 2024-09-10T18:40:18.364419+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:19.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:19 smithi151 bash[22565]: audit 2024-09-10T18:40:18.366054+0000 mon.a (mon.0) 1657 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72633-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[18597]: audit 2024-09-10T18:40:18.346779+0000 mon.b (mon.1) 125 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[18597]: audit 2024-09-10T18:40:18.359276+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[18597]: audit 2024-09-10T18:40:18.359455+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[18597]: cluster 2024-09-10T18:40:18.359719+0000 mon.a (mon.0) 1654 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:40:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[18597]: audit 2024-09-10T18:40:18.363826+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[18597]: audit 2024-09-10T18:40:18.364419+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[18597]: audit 2024-09-10T18:40:18.366054+0000 mon.a (mon.0) 1657 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72633-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[31607]: audit 2024-09-10T18:40:18.346779+0000 mon.b (mon.1) 125 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[31607]: audit 2024-09-10T18:40:18.359276+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-72601-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[31607]: audit 2024-09-10T18:40:18.359455+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[31607]: cluster 2024-09-10T18:40:18.359719+0000 mon.a (mon.0) 1654 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[31607]: audit 2024-09-10T18:40:18.363826+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[31607]: audit 2024-09-10T18:40:18.364419+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:19 smithi100 bash[31607]: audit 2024-09-10T18:40:18.366054+0000 mon.a (mon.0) 1657 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72633-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:20.392 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 1 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 10 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 13 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 7 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 14 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 0 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 15 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 11 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 5 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 8 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 6 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 3 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 4 0 2024-09-10T18:40:20.393 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 12 0 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 9 0 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2 0 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (91 ms) 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 3 tests from LibRadosListEC (1171 ms total) 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3041 ms) 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 1 test from LibRadosListNP (3041 ms total) 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] Global test environment tear-down 2024-09-10T18:40:20.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [==========] 11 tests from 3 test suites ran. (126061 ms total) 2024-09-10T18:40:20.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ PASSED ] 11 tests. 2024-09-10T18:40:20.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: audit 2024-09-10T18:40:19.357288+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]: dispatch 2024-09-10T18:40:20.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: audit 2024-09-10T18:40:19.364105+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]': finished 2024-09-10T18:40:20.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: audit 2024-09-10T18:40:19.364207+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:20.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: audit 2024-09-10T18:40:19.364343+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72633-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:20.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: cluster 2024-09-10T18:40:19.364453+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:40:20.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: audit 2024-09-10T18:40:19.364918+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:20.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: audit 2024-09-10T18:40:19.377971+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]: dispatch 2024-09-10T18:40:20.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: audit 2024-09-10T18:40:19.400911+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-72633-3","pool2":"test-rados-api-smithi100-72633-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:40:20.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: cluster 2024-09-10T18:40:19.858445+0000 mgr.y (mgr.24512) 254 : cluster [DBG] pgmap v277: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:20.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:20 smithi151 bash[22565]: cluster 2024-09-10T18:40:20.302021+0000 mon.a (mon.0) 1665 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:20.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: audit 2024-09-10T18:40:19.357288+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]: dispatch 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: audit 2024-09-10T18:40:19.364105+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]': finished 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: audit 2024-09-10T18:40:19.364207+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: audit 2024-09-10T18:40:19.364343+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72633-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: cluster 2024-09-10T18:40:19.364453+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: audit 2024-09-10T18:40:19.364918+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: audit 2024-09-10T18:40:19.377971+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]: dispatch 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: audit 2024-09-10T18:40:19.400911+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-72633-3","pool2":"test-rados-api-smithi100-72633-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: cluster 2024-09-10T18:40:19.858445+0000 mgr.y (mgr.24512) 254 : cluster [DBG] pgmap v277: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[18597]: cluster 2024-09-10T18:40:20.302021+0000 mon.a (mon.0) 1665 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: audit 2024-09-10T18:40:19.357288+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]: dispatch 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: audit 2024-09-10T18:40:19.364105+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-19"}]': finished 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: audit 2024-09-10T18:40:19.364207+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: audit 2024-09-10T18:40:19.364343+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72633-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: cluster 2024-09-10T18:40:19.364453+0000 mon.a (mon.0) 1661 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: audit 2024-09-10T18:40:19.364918+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]: dispatch 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: audit 2024-09-10T18:40:19.377971+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]: dispatch 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: audit 2024-09-10T18:40:19.400911+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-72633-3","pool2":"test-rados-api-smithi100-72633-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: cluster 2024-09-10T18:40:19.858445+0000 mgr.y (mgr.24512) 254 : cluster [DBG] pgmap v277: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:20.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:20 smithi100 bash[31607]: cluster 2024-09-10T18:40:20.302021+0000 mon.a (mon.0) 1665 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:21.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: cluster 2024-09-10T18:40:20.361766+0000 mon.a (mon.0) 1666 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:21.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.380924+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:21.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.381051+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]': finished 2024-09-10T18:40:21.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.381180+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-72633-3","pool2":"test-rados-api-smithi100-72633-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: cluster 2024-09-10T18:40:20.381286+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.388839+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-72601-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.409941+0000 mon.b (mon.1) 127 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.414600+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.414886+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.415800+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.416118+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.416615+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.416902+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:21.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:21 smithi151 bash[22565]: audit 2024-09-10T18:40:20.426495+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: cluster 2024-09-10T18:40:20.361766+0000 mon.a (mon.0) 1666 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.380924+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.381051+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]': finished 2024-09-10T18:40:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.381180+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-72633-3","pool2":"test-rados-api-smithi100-72633-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:40:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: cluster 2024-09-10T18:40:20.381286+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:40:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.388839+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-72601-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.409941+0000 mon.b (mon.1) 127 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.414600+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.414886+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.415800+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.416118+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.416615+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.416902+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18597]: audit 2024-09-10T18:40:20.426495+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: cluster 2024-09-10T18:40:20.361766+0000 mon.a (mon.0) 1666 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.380924+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? 172.21.15.100:0/727653266' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-72593-30"}]': finished 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.381051+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-19", "mode": "writeback"}]': finished 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.381180+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? 172.21.15.100:0/2468665862' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-72633-3","pool2":"test-rados-api-smithi100-72633-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: cluster 2024-09-10T18:40:20.381286+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.388839+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-72601-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.409941+0000 mon.b (mon.1) 127 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.414600+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.414886+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.415800+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.416118+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.416615+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.416902+0000 mon.a (mon.0) 1674 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[31607]: audit 2024-09-10T18:40:20.426495+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:21.796 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:40:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:21] "GET /metrics HTTP/1.1" 200 234742 "" "Prometheus/2.43.0" 2024-09-10T18:40:22.428 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:40:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.369352+0000 mon.b (mon.1) 128 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:22.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.382714+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-72601-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.383092+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.383218+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: cluster 2024-09-10T18:40:21.383303+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.385719+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.386518+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.387315+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: audit 2024-09-10T18:40:21.392715+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi100-72601-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:40:22.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:22 smithi151 bash[22565]: cluster 2024-09-10T18:40:21.859024+0000 mgr.y (mgr.24512) 255 : cluster [DBG] pgmap v280: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.369352+0000 mon.b (mon.1) 128 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.382714+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-72601-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.383092+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.383218+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: cluster 2024-09-10T18:40:21.383303+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.385719+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.386518+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.387315+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: audit 2024-09-10T18:40:21.392715+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi100-72601-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[18597]: cluster 2024-09-10T18:40:21.859024+0000 mgr.y (mgr.24512) 255 : cluster [DBG] pgmap v280: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.369352+0000 mon.b (mon.1) 128 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.382714+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-72601-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.383092+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-72593-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.383218+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: cluster 2024-09-10T18:40:21.383303+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:40:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.385719+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:22.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.386518+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:22.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.387315+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]: dispatch 2024-09-10T18:40:22.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: audit 2024-09-10T18:40:21.392715+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi100-72601-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:40:22.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:22 smithi100 bash[31607]: cluster 2024-09-10T18:40:21.859024+0000 mgr.y (mgr.24512) 255 : cluster [DBG] pgmap v280: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:22.834 INFO:tasks.workunit.client.0.smithi100.stdout:o.RoundTripAppendPP (2977 ms) 2024-09-10T18:40:22.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-10T18:40:22.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3015 ms) 2024-09-10T18:40:22.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-10T18:40:22.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3020 ms) 2024-09-10T18:40:22.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-10T18:40:22.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3024 ms) 2024-09-10T18:40:22.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-10T18:40:22.835 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: setting pool EIO 2024-09-10T18:40:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:23 smithi151 bash[22565]: audit 2024-09-10T18:40:22.175011+0000 mgr.y (mgr.24512) 256 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:23 smithi151 bash[22565]: cluster 2024-09-10T18:40:22.379254+0000 mon.a (mon.0) 1683 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:23 smithi151 bash[22565]: audit 2024-09-10T18:40:22.834279+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]': finished 2024-09-10T18:40:23.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:23 smithi151 bash[22565]: audit 2024-09-10T18:40:22.834501+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi100-72601-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:40:23.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:23 smithi151 bash[22565]: cluster 2024-09-10T18:40:22.834652+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:40:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[18597]: audit 2024-09-10T18:40:22.175011+0000 mgr.y (mgr.24512) 256 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[18597]: cluster 2024-09-10T18:40:22.379254+0000 mon.a (mon.0) 1683 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[18597]: audit 2024-09-10T18:40:22.834279+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]': finished 2024-09-10T18:40:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[18597]: audit 2024-09-10T18:40:22.834501+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi100-72601-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:40:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[18597]: cluster 2024-09-10T18:40:22.834652+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:40:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[31607]: audit 2024-09-10T18:40:22.175011+0000 mgr.y (mgr.24512) 256 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[31607]: cluster 2024-09-10T18:40:22.379254+0000 mon.a (mon.0) 1683 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[31607]: audit 2024-09-10T18:40:22.834279+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-19"}]': finished 2024-09-10T18:40:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[31607]: audit 2024-09-10T18:40:22.834501+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? 172.21.15.100:0/1035357708' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi100-72601-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:40:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:23 smithi100 bash[31607]: cluster 2024-09-10T18:40:22.834652+0000 mon.a (mon.0) 1686 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:40:24.678 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:40:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:24 smithi151 bash[22565]: audit 2024-09-10T18:40:23.398910+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:24 smithi151 bash[22565]: cluster 2024-09-10T18:40:23.399071+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:40:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:24 smithi151 bash[22565]: cluster 2024-09-10T18:40:23.859803+0000 mgr.y (mgr.24512) 257 : cluster [DBG] pgmap v283: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:24.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:24 smithi151 bash[22565]: audit 2024-09-10T18:40:24.325091+0000 mon.c (mon.2) 282 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:24.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:24 smithi151 bash[22565]: cluster 2024-09-10T18:40:24.401652+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:40:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[18597]: audit 2024-09-10T18:40:23.398910+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[18597]: cluster 2024-09-10T18:40:23.399071+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:40:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[18597]: cluster 2024-09-10T18:40:23.859803+0000 mgr.y (mgr.24512) 257 : cluster [DBG] pgmap v283: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[18597]: audit 2024-09-10T18:40:24.325091+0000 mon.c (mon.2) 282 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[18597]: cluster 2024-09-10T18:40:24.401652+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:40:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[31607]: audit 2024-09-10T18:40:23.398910+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-72593-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[31607]: cluster 2024-09-10T18:40:23.399071+0000 mon.a (mon.0) 1688 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:40:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[31607]: cluster 2024-09-10T18:40:23.859803+0000 mgr.y (mgr.24512) 257 : cluster [DBG] pgmap v283: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[31607]: audit 2024-09-10T18:40:24.325091+0000 mon.c (mon.2) 282 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:24 smithi100 bash[31607]: cluster 2024-09-10T18:40:24.401652+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:40:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:25 smithi151 bash[22565]: audit 2024-09-10T18:40:24.390681+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:25 smithi151 bash[22565]: audit 2024-09-10T18:40:24.407668+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:25 smithi151 bash[22565]: cluster 2024-09-10T18:40:25.303613+0000 mon.a (mon.0) 1691 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:25 smithi151 bash[22565]: audit 2024-09-10T18:40:25.408079+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:25 smithi151 bash[22565]: cluster 2024-09-10T18:40:25.409833+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:40:25.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[18597]: audit 2024-09-10T18:40:24.390681+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:25.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[18597]: audit 2024-09-10T18:40:24.407668+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:25.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[18597]: cluster 2024-09-10T18:40:25.303613+0000 mon.a (mon.0) 1691 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:25.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[18597]: audit 2024-09-10T18:40:25.408079+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:25.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[18597]: cluster 2024-09-10T18:40:25.409833+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:40:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[31607]: audit 2024-09-10T18:40:24.390681+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[31607]: audit 2024-09-10T18:40:24.407668+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[31607]: cluster 2024-09-10T18:40:25.303613+0000 mon.a (mon.0) 1691 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[31607]: audit 2024-09-10T18:40:25.408079+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:25 smithi100 bash[31607]: cluster 2024-09-10T18:40:25.409833+0000 mon.a (mon.0) 1693 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:40:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:25.412269+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:25.414910+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:25.415139+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.100:0/1785288549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-72601-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:25.422810+0000 mon.b (mon.1) 130 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:25.439820+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: cluster 2024-09-10T18:40:25.860672+0000 mgr.y (mgr.24512) 258 : cluster [DBG] pgmap v286: 260 pgs: 32 creating+peering, 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:26.394279+0000 mon.b (mon.1) 131 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:26.406835+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:26.407070+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.100:0/1785288549' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-72601-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:26.407525+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: cluster 2024-09-10T18:40:26.407687+0000 mon.a (mon.0) 1700 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:26.410214+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:26.413276+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[18597]: audit 2024-09-10T18:40:26.413621+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:25.412269+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:25.414910+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:25.415139+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.100:0/1785288549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-72601-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:25.422810+0000 mon.b (mon.1) 130 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:25.439820+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: cluster 2024-09-10T18:40:25.860672+0000 mgr.y (mgr.24512) 258 : cluster [DBG] pgmap v286: 260 pgs: 32 creating+peering, 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:26.394279+0000 mon.b (mon.1) 131 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:26.406835+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:26.407070+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.100:0/1785288549' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-72601-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:26.407525+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:26.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: cluster 2024-09-10T18:40:26.407687+0000 mon.a (mon.0) 1700 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:40:26.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:26.410214+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:26.413276+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:26.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:26 smithi100 bash[31607]: audit 2024-09-10T18:40:26.413621+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:25.412269+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:25.414910+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:25.415139+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.100:0/1785288549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-72601-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:25.422810+0000 mon.b (mon.1) 130 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:25.439820+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: cluster 2024-09-10T18:40:25.860672+0000 mgr.y (mgr.24512) 258 : cluster [DBG] pgmap v286: 260 pgs: 32 creating+peering, 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:26.394279+0000 mon.b (mon.1) 131 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:26.406835+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:26.407070+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? 172.21.15.100:0/1785288549' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-72601-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:26.407525+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: cluster 2024-09-10T18:40:26.407687+0000 mon.a (mon.0) 1700 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:40:26.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:26.410214+0000 mon.c (mon.2) 284 : audit [INF] from='client.? 172.21.15.100:0/3263365947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:26.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:26.413276+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:26.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:26 smithi151 bash[22565]: audit 2024-09-10T18:40:26.413621+0000 mon.a (mon.0) 1702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]: dispatch 2024-09-10T18:40:28.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: audit 2024-09-10T18:40:27.398227+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]: dispatch 2024-09-10T18:40:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: audit 2024-09-10T18:40:27.413010+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]': finished 2024-09-10T18:40:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: audit 2024-09-10T18:40:27.413226+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: cluster 2024-09-10T18:40:27.413343+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:40:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: audit 2024-09-10T18:40:27.415139+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]: dispatch 2024-09-10T18:40:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: audit 2024-09-10T18:40:27.441076+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: audit 2024-09-10T18:40:27.861025+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: cluster 2024-09-10T18:40:27.861137+0000 mgr.y (mgr.24512) 259 : cluster [DBG] pgmap v289: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:28 smithi151 bash[22565]: audit 2024-09-10T18:40:27.863724+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-72593-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: audit 2024-09-10T18:40:27.398227+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]: dispatch 2024-09-10T18:40:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: audit 2024-09-10T18:40:27.413010+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]': finished 2024-09-10T18:40:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: audit 2024-09-10T18:40:27.413226+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: cluster 2024-09-10T18:40:27.413343+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: audit 2024-09-10T18:40:27.415139+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]: dispatch 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: audit 2024-09-10T18:40:27.441076+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: audit 2024-09-10T18:40:27.861025+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: cluster 2024-09-10T18:40:27.861137+0000 mgr.y (mgr.24512) 259 : cluster [DBG] pgmap v289: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[18597]: audit 2024-09-10T18:40:27.863724+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-72593-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: audit 2024-09-10T18:40:27.398227+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]: dispatch 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: audit 2024-09-10T18:40:27.413010+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-21"}]': finished 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: audit 2024-09-10T18:40:27.413226+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-72593-31"}]': finished 2024-09-10T18:40:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: cluster 2024-09-10T18:40:27.413343+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:40:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: audit 2024-09-10T18:40:27.415139+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]: dispatch 2024-09-10T18:40:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: audit 2024-09-10T18:40:27.441076+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: audit 2024-09-10T18:40:27.861025+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: cluster 2024-09-10T18:40:27.861137+0000 mgr.y (mgr.24512) 259 : cluster [DBG] pgmap v289: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:40:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:28 smithi100 bash[31607]: audit 2024-09-10T18:40:27.863724+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-72593-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:29.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: cluster 2024-09-10T18:40:28.406092+0000 mon.a (mon.0) 1710 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:29.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: audit 2024-09-10T18:40:28.429645+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]': finished 2024-09-10T18:40:29.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: audit 2024-09-10T18:40:28.429897+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-72593-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:29.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: cluster 2024-09-10T18:40:28.430027+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:40:29.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: audit 2024-09-10T18:40:28.433768+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-72593-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:29.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: audit 2024-09-10T18:40:28.440068+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.100:0/1580852781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:29.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: audit 2024-09-10T18:40:28.440583+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:29.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: audit 2024-09-10T18:40:28.481867+0000 mon.b (mon.1) 133 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:29.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:29 smithi151 bash[22565]: audit 2024-09-10T18:40:28.498850+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: cluster 2024-09-10T18:40:28.406092+0000 mon.a (mon.0) 1710 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: audit 2024-09-10T18:40:28.429645+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]': finished 2024-09-10T18:40:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: audit 2024-09-10T18:40:28.429897+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-72593-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: cluster 2024-09-10T18:40:28.430027+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: audit 2024-09-10T18:40:28.433768+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-72593-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: audit 2024-09-10T18:40:28.440068+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.100:0/1580852781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: audit 2024-09-10T18:40:28.440583+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: audit 2024-09-10T18:40:28.481867+0000 mon.b (mon.1) 133 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[18597]: audit 2024-09-10T18:40:28.498850+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: cluster 2024-09-10T18:40:28.406092+0000 mon.a (mon.0) 1710 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: audit 2024-09-10T18:40:28.429645+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-21", "mode": "writeback"}]': finished 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: audit 2024-09-10T18:40:28.429897+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-72593-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: cluster 2024-09-10T18:40:28.430027+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: audit 2024-09-10T18:40:28.433768+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-72593-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: audit 2024-09-10T18:40:28.440068+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.100:0/1580852781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: audit 2024-09-10T18:40:28.440583+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: audit 2024-09-10T18:40:28.481867+0000 mon.b (mon.1) 133 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:29 smithi100 bash[31607]: audit 2024-09-10T18:40:28.498850+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[18597]: audit 2024-09-10T18:40:29.849745+0000 mon.b (mon.1) 134 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[18597]: cluster 2024-09-10T18:40:29.861901+0000 mgr.y (mgr.24512) 260 : cluster [DBG] pgmap v292: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[18597]: audit 2024-09-10T18:40:29.864364+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[18597]: audit 2024-09-10T18:40:29.864681+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[18597]: cluster 2024-09-10T18:40:29.864892+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:40:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[18597]: audit 2024-09-10T18:40:29.872618+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[18597]: cluster 2024-09-10T18:40:30.305777+0000 mon.a (mon.0) 1721 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[31607]: audit 2024-09-10T18:40:29.849745+0000 mon.b (mon.1) 134 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[31607]: cluster 2024-09-10T18:40:29.861901+0000 mgr.y (mgr.24512) 260 : cluster [DBG] pgmap v292: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[31607]: audit 2024-09-10T18:40:29.864364+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[31607]: audit 2024-09-10T18:40:29.864681+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[31607]: cluster 2024-09-10T18:40:29.864892+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:40:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[31607]: audit 2024-09-10T18:40:29.872618+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:30 smithi100 bash[31607]: cluster 2024-09-10T18:40:30.305777+0000 mon.a (mon.0) 1721 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:30 smithi151 bash[22565]: audit 2024-09-10T18:40:29.849745+0000 mon.b (mon.1) 134 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:30 smithi151 bash[22565]: cluster 2024-09-10T18:40:29.861901+0000 mgr.y (mgr.24512) 260 : cluster [DBG] pgmap v292: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:30 smithi151 bash[22565]: audit 2024-09-10T18:40:29.864364+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-72601-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:30.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:30 smithi151 bash[22565]: audit 2024-09-10T18:40:29.864681+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:30 smithi151 bash[22565]: cluster 2024-09-10T18:40:29.864892+0000 mon.a (mon.0) 1719 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:40:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:30 smithi151 bash[22565]: audit 2024-09-10T18:40:29.872618+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]: dispatch 2024-09-10T18:40:30.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:30 smithi151 bash[22565]: cluster 2024-09-10T18:40:30.305777+0000 mon.a (mon.0) 1721 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[18597]: cluster 2024-09-10T18:40:30.426308+0000 mon.a (mon.0) 1722 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[18597]: audit 2024-09-10T18:40:30.449441+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-72593-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[18597]: audit 2024-09-10T18:40:30.449578+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]': finished 2024-09-10T18:40:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[18597]: cluster 2024-09-10T18:40:30.449766+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:40:31.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:31] "GET /metrics HTTP/1.1" 200 230854 "" "Prometheus/2.43.0" 2024-09-10T18:40:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[31607]: cluster 2024-09-10T18:40:30.426308+0000 mon.a (mon.0) 1722 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[31607]: audit 2024-09-10T18:40:30.449441+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-72593-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[31607]: audit 2024-09-10T18:40:30.449578+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]': finished 2024-09-10T18:40:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:31 smithi100 bash[31607]: cluster 2024-09-10T18:40:30.449766+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:40:31.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:31 smithi151 bash[22565]: cluster 2024-09-10T18:40:30.426308+0000 mon.a (mon.0) 1722 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:31.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:31 smithi151 bash[22565]: audit 2024-09-10T18:40:30.449441+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-72593-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:31.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:31 smithi151 bash[22565]: audit 2024-09-10T18:40:30.449578+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-21"}]': finished 2024-09-10T18:40:31.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:31 smithi151 bash[22565]: cluster 2024-09-10T18:40:30.449766+0000 mon.a (mon.0) 1725 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:40:32.474 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:40:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:40:32.474 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:32 smithi151 bash[22565]: cluster 2024-09-10T18:40:31.448783+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:40:32.475 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:32 smithi151 bash[22565]: audit 2024-09-10T18:40:31.468269+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.100:0/2361253846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:32.475 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:32 smithi151 bash[22565]: cluster 2024-09-10T18:40:31.862690+0000 mgr.y (mgr.24512) 261 : cluster [DBG] pgmap v295: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:40:32.475 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:32 smithi151 bash[22565]: audit 2024-09-10T18:40:32.453897+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.100:0/2361253846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:32.475 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:32 smithi151 bash[22565]: cluster 2024-09-10T18:40:32.454464+0000 mon.a (mon.0) 1729 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:40:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[18597]: cluster 2024-09-10T18:40:31.448783+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:40:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[18597]: audit 2024-09-10T18:40:31.468269+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.100:0/2361253846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[18597]: cluster 2024-09-10T18:40:31.862690+0000 mgr.y (mgr.24512) 261 : cluster [DBG] pgmap v295: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:40:32.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[18597]: audit 2024-09-10T18:40:32.453897+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.100:0/2361253846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:32.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[18597]: cluster 2024-09-10T18:40:32.454464+0000 mon.a (mon.0) 1729 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:40:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[31607]: cluster 2024-09-10T18:40:31.448783+0000 mon.a (mon.0) 1726 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:40:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[31607]: audit 2024-09-10T18:40:31.468269+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.100:0/2361253846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[31607]: cluster 2024-09-10T18:40:31.862690+0000 mgr.y (mgr.24512) 261 : cluster [DBG] pgmap v295: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:40:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[31607]: audit 2024-09-10T18:40:32.453897+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.100:0/2361253846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:32.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:32 smithi100 bash[31607]: cluster 2024-09-10T18:40:32.454464+0000 mon.a (mon.0) 1729 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:40:33.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:32.186445+0000 mgr.y (mgr.24512) 262 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:32.441916+0000 mon.b (mon.1) 135 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:32.460853+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:32.461178+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:33.460842+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:33.461210+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: cluster 2024-09-10T18:40:33.462709+0000 mon.a (mon.0) 1734 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:33.464077+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.100:0/3531842000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:33.465619+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[31607]: audit 2024-09-10T18:40:33.466107+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:32.186445+0000 mgr.y (mgr.24512) 262 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:32.441916+0000 mon.b (mon.1) 135 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:32.460853+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:32.461178+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:33.460842+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:33.461210+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: cluster 2024-09-10T18:40:33.462709+0000 mon.a (mon.0) 1734 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:33.464077+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.100:0/3531842000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:33.465619+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:33.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:33 smithi100 bash[18597]: audit 2024-09-10T18:40:33.466107+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:32.186445+0000 mgr.y (mgr.24512) 262 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:32.441916+0000 mon.b (mon.1) 135 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:32.460853+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:32.461178+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:33.460842+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:33.461210+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: cluster 2024-09-10T18:40:33.462709+0000 mon.a (mon.0) 1734 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:33.464077+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.100:0/3531842000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:33.465619+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]: dispatch 2024-09-10T18:40:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:33 smithi151 bash[22565]: audit 2024-09-10T18:40:33.466107+0000 mon.a (mon.0) 1736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:34.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[18597]: cluster 2024-09-10T18:40:33.863537+0000 mgr.y (mgr.24512) 263 : cluster [DBG] pgmap v298: 292 pgs: 96 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:40:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[18597]: audit 2024-09-10T18:40:34.459989+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[18597]: audit 2024-09-10T18:40:34.460161+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[18597]: cluster 2024-09-10T18:40:34.460485+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:40:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[31607]: cluster 2024-09-10T18:40:33.863537+0000 mgr.y (mgr.24512) 263 : cluster [DBG] pgmap v298: 292 pgs: 96 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:40:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[31607]: audit 2024-09-10T18:40:34.459989+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[31607]: audit 2024-09-10T18:40:34.460161+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:34 smithi100 bash[31607]: cluster 2024-09-10T18:40:34.460485+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:40:34.928 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:40:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:34.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:34 smithi151 bash[22565]: cluster 2024-09-10T18:40:33.863537+0000 mgr.y (mgr.24512) 263 : cluster [DBG] pgmap v298: 292 pgs: 96 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:40:34.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:34 smithi151 bash[22565]: audit 2024-09-10T18:40:34.459989+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.100:0/4108573684' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-72593-32"}]': finished 2024-09-10T18:40:34.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:34 smithi151 bash[22565]: audit 2024-09-10T18:40:34.460161+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:34.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:34 smithi151 bash[22565]: cluster 2024-09-10T18:40:34.460485+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:40:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:34.480849+0000 mon.b (mon.1) 136 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:34.497601+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:34.502710+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:34.504153+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:34.504539+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-72593-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: cluster 2024-09-10T18:40:35.307442+0000 mon.a (mon.0) 1744 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:35.451803+0000 mon.b (mon.1) 137 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:35.466719+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:35.467277+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-72593-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: cluster 2024-09-10T18:40:35.467603+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:35.470930+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-72593-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:35.471629+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[18597]: audit 2024-09-10T18:40:35.473592+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? 172.21.15.100:0/57254007' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:34.480849+0000 mon.b (mon.1) 136 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:34.497601+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:34.502710+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:34.504153+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:34.504539+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-72593-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: cluster 2024-09-10T18:40:35.307442+0000 mon.a (mon.0) 1744 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:35.451803+0000 mon.b (mon.1) 137 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:35.466719+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:35.467277+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-72593-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: cluster 2024-09-10T18:40:35.467603+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:35.470930+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-72593-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:35.471629+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:35.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:35 smithi100 bash[31607]: audit 2024-09-10T18:40:35.473592+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? 172.21.15.100:0/57254007' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:35.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:34.480849+0000 mon.b (mon.1) 136 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:34.497601+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:34.502710+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:34.504153+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:34.504539+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-72593-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: cluster 2024-09-10T18:40:35.307442+0000 mon.a (mon.0) 1744 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:35.451803+0000 mon.b (mon.1) 137 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:35.466719+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:35.467277+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-72593-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: cluster 2024-09-10T18:40:35.467603+0000 mon.a (mon.0) 1747 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:35.470930+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-72593-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:35.471629+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:35 smithi151 bash[22565]: audit 2024-09-10T18:40:35.473592+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? 172.21.15.100:0/57254007' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:37.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:36 smithi151 bash[22565]: cluster 2024-09-10T18:40:35.864413+0000 mgr.y (mgr.24512) 264 : cluster [DBG] pgmap v301: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:40:37.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:36 smithi151 bash[22565]: audit 2024-09-10T18:40:35.894006+0000 mon.c (mon.2) 287 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:36 smithi151 bash[22565]: audit 2024-09-10T18:40:36.343225+0000 mon.c (mon.2) 288 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:37.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:36 smithi151 bash[22565]: audit 2024-09-10T18:40:36.344824+0000 mon.c (mon.2) 289 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:37.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:36 smithi151 bash[22565]: audit 2024-09-10T18:40:36.355513+0000 mon.a (mon.0) 1751 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[18597]: cluster 2024-09-10T18:40:35.864413+0000 mgr.y (mgr.24512) 264 : cluster [DBG] pgmap v301: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:40:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[18597]: audit 2024-09-10T18:40:35.894006+0000 mon.c (mon.2) 287 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[18597]: audit 2024-09-10T18:40:36.343225+0000 mon.c (mon.2) 288 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[18597]: audit 2024-09-10T18:40:36.344824+0000 mon.c (mon.2) 289 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[18597]: audit 2024-09-10T18:40:36.355513+0000 mon.a (mon.0) 1751 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[31607]: cluster 2024-09-10T18:40:35.864413+0000 mgr.y (mgr.24512) 264 : cluster [DBG] pgmap v301: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:40:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[31607]: audit 2024-09-10T18:40:35.894006+0000 mon.c (mon.2) 287 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[31607]: audit 2024-09-10T18:40:36.343225+0000 mon.c (mon.2) 288 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[31607]: audit 2024-09-10T18:40:36.344824+0000 mon.c (mon.2) 289 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:36 smithi100 bash[31607]: audit 2024-09-10T18:40:36.355513+0000 mon.a (mon.0) 1751 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:38.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: audit 2024-09-10T18:40:36.873860+0000 mon.b (mon.1) 138 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]: dispatch 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: audit 2024-09-10T18:40:36.888511+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]': finished 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: audit 2024-09-10T18:40:36.889065+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? 172.21.15.100:0/57254007' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: cluster 2024-09-10T18:40:36.889522+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: audit 2024-09-10T18:40:36.892176+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]: dispatch 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: cluster 2024-09-10T18:40:37.460523+0000 mon.a (mon.0) 1756 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: audit 2024-09-10T18:40:37.471186+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-72593-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: audit 2024-09-10T18:40:37.471679+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]': finished 2024-09-10T18:40:38.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:37 smithi151 bash[22565]: cluster 2024-09-10T18:40:37.471808+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:40:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: audit 2024-09-10T18:40:36.873860+0000 mon.b (mon.1) 138 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]: dispatch 2024-09-10T18:40:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: audit 2024-09-10T18:40:36.888511+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]': finished 2024-09-10T18:40:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: audit 2024-09-10T18:40:36.889065+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? 172.21.15.100:0/57254007' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: cluster 2024-09-10T18:40:36.889522+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: audit 2024-09-10T18:40:36.892176+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]: dispatch 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: cluster 2024-09-10T18:40:37.460523+0000 mon.a (mon.0) 1756 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: audit 2024-09-10T18:40:37.471186+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-72593-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: audit 2024-09-10T18:40:37.471679+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]': finished 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[31607]: cluster 2024-09-10T18:40:37.471808+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: audit 2024-09-10T18:40:36.873860+0000 mon.b (mon.1) 138 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]: dispatch 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: audit 2024-09-10T18:40:36.888511+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-23"}]': finished 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: audit 2024-09-10T18:40:36.889065+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? 172.21.15.100:0/57254007' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-72601-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: cluster 2024-09-10T18:40:36.889522+0000 mon.a (mon.0) 1754 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:40:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: audit 2024-09-10T18:40:36.892176+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]: dispatch 2024-09-10T18:40:38.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: cluster 2024-09-10T18:40:37.460523+0000 mon.a (mon.0) 1756 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:38.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: audit 2024-09-10T18:40:37.471186+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-72593-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:38.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: audit 2024-09-10T18:40:37.471679+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-23", "mode": "writeback"}]': finished 2024-09-10T18:40:38.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:37 smithi100 bash[18597]: cluster 2024-09-10T18:40:37.471808+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:40:39.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:38 smithi151 bash[22565]: cluster 2024-09-10T18:40:37.865418+0000 mgr.y (mgr.24512) 265 : cluster [DBG] pgmap v304: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:40:39.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:38 smithi151 bash[22565]: cluster 2024-09-10T18:40:38.479868+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:40:39.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:38 smithi151 bash[22565]: audit 2024-09-10T18:40:38.503146+0000 mon.b (mon.1) 139 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:39.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:38 smithi151 bash[22565]: audit 2024-09-10T18:40:38.519981+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[31607]: cluster 2024-09-10T18:40:37.865418+0000 mgr.y (mgr.24512) 265 : cluster [DBG] pgmap v304: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:40:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[31607]: cluster 2024-09-10T18:40:38.479868+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:40:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[31607]: audit 2024-09-10T18:40:38.503146+0000 mon.b (mon.1) 139 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[31607]: audit 2024-09-10T18:40:38.519981+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[18597]: cluster 2024-09-10T18:40:37.865418+0000 mgr.y (mgr.24512) 265 : cluster [DBG] pgmap v304: 300 pgs: 8 unknown, 292 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:40:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[18597]: cluster 2024-09-10T18:40:38.479868+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:40:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[18597]: audit 2024-09-10T18:40:38.503146+0000 mon.b (mon.1) 139 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:38 smithi100 bash[18597]: audit 2024-09-10T18:40:38.519981+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:40.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: audit 2024-09-10T18:40:39.332158+0000 mon.a (mon.0) 1762 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:40.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: audit 2024-09-10T18:40:39.335403+0000 mon.c (mon.2) 290 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:40.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: audit 2024-09-10T18:40:39.468122+0000 mon.b (mon.1) 140 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:40.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: audit 2024-09-10T18:40:39.482478+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:40.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: cluster 2024-09-10T18:40:39.482677+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:40:40.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: audit 2024-09-10T18:40:39.484198+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:40.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: audit 2024-09-10T18:40:39.485267+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:40.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:40 smithi151 bash[22565]: cluster 2024-09-10T18:40:39.866204+0000 mgr.y (mgr.24512) 266 : cluster [DBG] pgmap v307: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: audit 2024-09-10T18:40:39.332158+0000 mon.a (mon.0) 1762 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: audit 2024-09-10T18:40:39.335403+0000 mon.c (mon.2) 290 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: audit 2024-09-10T18:40:39.468122+0000 mon.b (mon.1) 140 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: audit 2024-09-10T18:40:39.482478+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:40.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: cluster 2024-09-10T18:40:39.482677+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: audit 2024-09-10T18:40:39.484198+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: audit 2024-09-10T18:40:39.485267+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[31607]: cluster 2024-09-10T18:40:39.866204+0000 mgr.y (mgr.24512) 266 : cluster [DBG] pgmap v307: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: audit 2024-09-10T18:40:39.332158+0000 mon.a (mon.0) 1762 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: audit 2024-09-10T18:40:39.335403+0000 mon.c (mon.2) 290 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: audit 2024-09-10T18:40:39.468122+0000 mon.b (mon.1) 140 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: audit 2024-09-10T18:40:39.482478+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: cluster 2024-09-10T18:40:39.482677+0000 mon.a (mon.0) 1764 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: audit 2024-09-10T18:40:39.484198+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: audit 2024-09-10T18:40:39.485267+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]: dispatch 2024-09-10T18:40:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:40 smithi100 bash[18597]: cluster 2024-09-10T18:40:39.866204+0000 mgr.y (mgr.24512) 266 : cluster [DBG] pgmap v307: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:41.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: cluster 2024-09-10T18:40:40.475366+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:41.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: cluster 2024-09-10T18:40:40.477007+0000 mon.a (mon.0) 1768 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:41.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: audit 2024-09-10T18:40:40.492660+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:41.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: audit 2024-09-10T18:40:40.492902+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]': finished 2024-09-10T18:40:41.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: cluster 2024-09-10T18:40:40.493092+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:40:41.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: audit 2024-09-10T18:40:40.499326+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.100:0/3696494060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:41.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: audit 2024-09-10T18:40:40.500659+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:41.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:41 smithi151 bash[22565]: audit 2024-09-10T18:40:40.502105+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:41] "GET /metrics HTTP/1.1" 200 233335 "" "Prometheus/2.43.0" 2024-09-10T18:40:41.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: cluster 2024-09-10T18:40:40.475366+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:41.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: cluster 2024-09-10T18:40:40.477007+0000 mon.a (mon.0) 1768 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: audit 2024-09-10T18:40:40.492660+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: audit 2024-09-10T18:40:40.492902+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]': finished 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: cluster 2024-09-10T18:40:40.493092+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: audit 2024-09-10T18:40:40.499326+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.100:0/3696494060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: audit 2024-09-10T18:40:40.500659+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[18597]: audit 2024-09-10T18:40:40.502105+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: cluster 2024-09-10T18:40:40.475366+0000 mon.a (mon.0) 1767 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: cluster 2024-09-10T18:40:40.477007+0000 mon.a (mon.0) 1768 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: audit 2024-09-10T18:40:40.492660+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: audit 2024-09-10T18:40:40.492902+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-23"}]': finished 2024-09-10T18:40:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: cluster 2024-09-10T18:40:40.493092+0000 mon.a (mon.0) 1771 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:40:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: audit 2024-09-10T18:40:40.499326+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.100:0/3696494060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: audit 2024-09-10T18:40:40.500659+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]: dispatch 2024-09-10T18:40:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:41 smithi100 bash[31607]: audit 2024-09-10T18:40:40.502105+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:42.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:40:42.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.494797+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:42.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.494995+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: cluster 2024-09-10T18:40:41.495408+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.539842+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: cluster 2024-09-10T18:40:41.866965+0000 mgr.y (mgr.24512) 267 : cluster [DBG] pgmap v310: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.939789+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.940835+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.943128+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.944097+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[18597]: audit 2024-09-10T18:40:41.945577+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.494797+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.494995+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: cluster 2024-09-10T18:40:41.495408+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.539842+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: cluster 2024-09-10T18:40:41.866965+0000 mgr.y (mgr.24512) 267 : cluster [DBG] pgmap v310: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.939789+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.940835+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.943128+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.944097+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:42.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:42 smithi100 bash[31607]: audit 2024-09-10T18:40:41.945577+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.494797+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? 172.21.15.100:0/4228895930' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-72593-33"}]': finished 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.494995+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: cluster 2024-09-10T18:40:41.495408+0000 mon.a (mon.0) 1776 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.539842+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: cluster 2024-09-10T18:40:41.866965+0000 mgr.y (mgr.24512) 267 : cluster [DBG] pgmap v310: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.939789+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.940835+0000 mon.c (mon.2) 293 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.943128+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.944097+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:42 smithi151 bash[22565]: audit 2024-09-10T18:40:41.945577+0000 mon.a (mon.0) 1779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: audit 2024-09-10T18:40:42.197974+0000 mgr.y (mgr.24512) 268 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: audit 2024-09-10T18:40:42.503160+0000 mon.b (mon.1) 141 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: audit 2024-09-10T18:40:42.514294+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: cluster 2024-09-10T18:40:42.514449+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: audit 2024-09-10T18:40:42.519349+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? 172.21.15.100:0/2950931384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: audit 2024-09-10T18:40:42.519422+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: audit 2024-09-10T18:40:42.519826+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[31607]: audit 2024-09-10T18:40:42.521215+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: audit 2024-09-10T18:40:42.197974+0000 mgr.y (mgr.24512) 268 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: audit 2024-09-10T18:40:42.503160+0000 mon.b (mon.1) 141 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: audit 2024-09-10T18:40:42.514294+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: cluster 2024-09-10T18:40:42.514449+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: audit 2024-09-10T18:40:42.519349+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? 172.21.15.100:0/2950931384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: audit 2024-09-10T18:40:42.519422+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: audit 2024-09-10T18:40:42.519826+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:44.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:43 smithi100 bash[18597]: audit 2024-09-10T18:40:42.521215+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: audit 2024-09-10T18:40:42.197974+0000 mgr.y (mgr.24512) 268 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: audit 2024-09-10T18:40:42.503160+0000 mon.b (mon.1) 141 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: audit 2024-09-10T18:40:42.514294+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-72593-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: cluster 2024-09-10T18:40:42.514449+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:40:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: audit 2024-09-10T18:40:42.519349+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? 172.21.15.100:0/2950931384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: audit 2024-09-10T18:40:42.519422+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: audit 2024-09-10T18:40:42.519826+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:43 smithi151 bash[22565]: audit 2024-09-10T18:40:42.521215+0000 mon.a (mon.0) 1784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:44.928 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: audit 2024-09-10T18:40:43.515635+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.100:0/2950931384' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: cluster 2024-09-10T18:40:43.868537+0000 mgr.y (mgr.24512) 269 : cluster [DBG] pgmap v313: 292 pgs: 96 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: audit 2024-09-10T18:40:43.945590+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: cluster 2024-09-10T18:40:43.945945+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: audit 2024-09-10T18:40:44.524454+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: cluster 2024-09-10T18:40:44.524648+0000 mon.a (mon.0) 1789 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: audit 2024-09-10T18:40:44.527440+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: audit 2024-09-10T18:40:44.529615+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.100:0/4292802718' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: audit 2024-09-10T18:40:44.530867+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[18597]: audit 2024-09-10T18:40:44.544418+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: audit 2024-09-10T18:40:43.515635+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.100:0/2950931384' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: cluster 2024-09-10T18:40:43.868537+0000 mgr.y (mgr.24512) 269 : cluster [DBG] pgmap v313: 292 pgs: 96 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: audit 2024-09-10T18:40:43.945590+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: cluster 2024-09-10T18:40:43.945945+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:40:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: audit 2024-09-10T18:40:44.524454+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: cluster 2024-09-10T18:40:44.524648+0000 mon.a (mon.0) 1789 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:40:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: audit 2024-09-10T18:40:44.527440+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: audit 2024-09-10T18:40:44.529615+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.100:0/4292802718' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: audit 2024-09-10T18:40:44.530867+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:44 smithi100 bash[31607]: audit 2024-09-10T18:40:44.544418+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:45.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: audit 2024-09-10T18:40:43.515635+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.100:0/2950931384' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:45.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: cluster 2024-09-10T18:40:43.868537+0000 mgr.y (mgr.24512) 269 : cluster [DBG] pgmap v313: 292 pgs: 96 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:45.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: audit 2024-09-10T18:40:43.945590+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:45.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: cluster 2024-09-10T18:40:43.945945+0000 mon.a (mon.0) 1787 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:40:45.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: audit 2024-09-10T18:40:44.524454+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi100-72593-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:45.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: cluster 2024-09-10T18:40:44.524648+0000 mon.a (mon.0) 1789 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:40:45.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: audit 2024-09-10T18:40:44.527440+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: audit 2024-09-10T18:40:44.529615+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.100:0/4292802718' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: audit 2024-09-10T18:40:44.530867+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:44 smithi151 bash[22565]: audit 2024-09-10T18:40:44.544418+0000 mon.a (mon.0) 1791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:46 smithi151 bash[22565]: audit 2024-09-10T18:40:45.308476+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:46 smithi151 bash[22565]: audit 2024-09-10T18:40:45.322437+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:46 smithi151 bash[22565]: audit 2024-09-10T18:40:45.323067+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:46 smithi151 bash[22565]: cluster 2024-09-10T18:40:45.323235+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:40:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:46 smithi151 bash[22565]: audit 2024-09-10T18:40:45.327038+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:46.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:46 smithi151 bash[22565]: cluster 2024-09-10T18:40:45.869491+0000 mgr.y (mgr.24512) 270 : cluster [DBG] pgmap v316: 332 pgs: 40 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:40:46.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[31607]: audit 2024-09-10T18:40:45.308476+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:46.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[31607]: audit 2024-09-10T18:40:45.322437+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:46.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[31607]: audit 2024-09-10T18:40:45.323067+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:46.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[31607]: cluster 2024-09-10T18:40:45.323235+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[31607]: audit 2024-09-10T18:40:45.327038+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[31607]: cluster 2024-09-10T18:40:45.869491+0000 mgr.y (mgr.24512) 270 : cluster [DBG] pgmap v316: 332 pgs: 40 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[18597]: audit 2024-09-10T18:40:45.308476+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[18597]: audit 2024-09-10T18:40:45.322437+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[18597]: audit 2024-09-10T18:40:45.323067+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[18597]: cluster 2024-09-10T18:40:45.323235+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[18597]: audit 2024-09-10T18:40:45.327038+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:46 smithi100 bash[18597]: cluster 2024-09-10T18:40:45.869491+0000 mgr.y (mgr.24512) 270 : cluster [DBG] pgmap v316: 332 pgs: 40 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: cluster 2024-09-10T18:40:46.315026+0000 mon.a (mon.0) 1796 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: audit 2024-09-10T18:40:46.322309+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]: dispatch 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: audit 2024-09-10T18:40:46.332112+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]': finished 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: cluster 2024-09-10T18:40:46.332330+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: audit 2024-09-10T18:40:46.339088+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: audit 2024-09-10T18:40:46.340051+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.100:0/3629302979' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: audit 2024-09-10T18:40:46.340121+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]: dispatch 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: audit 2024-09-10T18:40:46.340354+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:47 smithi151 bash[22565]: audit 2024-09-10T18:40:46.341241+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:47.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: cluster 2024-09-10T18:40:46.315026+0000 mon.a (mon.0) 1796 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:47.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: audit 2024-09-10T18:40:46.322309+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]: dispatch 2024-09-10T18:40:47.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: audit 2024-09-10T18:40:46.332112+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]': finished 2024-09-10T18:40:47.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: cluster 2024-09-10T18:40:46.332330+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: audit 2024-09-10T18:40:46.339088+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: audit 2024-09-10T18:40:46.340051+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.100:0/3629302979' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: audit 2024-09-10T18:40:46.340121+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]: dispatch 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: audit 2024-09-10T18:40:46.340354+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[31607]: audit 2024-09-10T18:40:46.341241+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: cluster 2024-09-10T18:40:46.315026+0000 mon.a (mon.0) 1796 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: audit 2024-09-10T18:40:46.322309+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]: dispatch 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: audit 2024-09-10T18:40:46.332112+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-25"}]': finished 2024-09-10T18:40:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: cluster 2024-09-10T18:40:46.332330+0000 mon.a (mon.0) 1798 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:40:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: audit 2024-09-10T18:40:46.339088+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: audit 2024-09-10T18:40:46.340051+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.100:0/3629302979' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: audit 2024-09-10T18:40:46.340121+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]: dispatch 2024-09-10T18:40:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: audit 2024-09-10T18:40:46.340354+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:47 smithi100 bash[18597]: audit 2024-09-10T18:40:46.341241+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: cluster 2024-09-10T18:40:47.327656+0000 mon.a (mon.0) 1802 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: audit 2024-09-10T18:40:47.338863+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]': finished 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: audit 2024-09-10T18:40:47.339010+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: audit 2024-09-10T18:40:47.339190+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: cluster 2024-09-10T18:40:47.339476+0000 mon.a (mon.0) 1806 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: audit 2024-09-10T18:40:47.341140+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: audit 2024-09-10T18:40:47.343397+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: audit 2024-09-10T18:40:47.364772+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: audit 2024-09-10T18:40:47.381803+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:48 smithi151 bash[22565]: cluster 2024-09-10T18:40:47.870351+0000 mgr.y (mgr.24512) 271 : cluster [DBG] pgmap v319: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:40:48.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: cluster 2024-09-10T18:40:47.327656+0000 mon.a (mon.0) 1802 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:48.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: audit 2024-09-10T18:40:47.338863+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]': finished 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: audit 2024-09-10T18:40:47.339010+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: audit 2024-09-10T18:40:47.339190+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: cluster 2024-09-10T18:40:47.339476+0000 mon.a (mon.0) 1806 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: audit 2024-09-10T18:40:47.341140+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: audit 2024-09-10T18:40:47.343397+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: audit 2024-09-10T18:40:47.364772+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: audit 2024-09-10T18:40:47.381803+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[18597]: cluster 2024-09-10T18:40:47.870351+0000 mgr.y (mgr.24512) 271 : cluster [DBG] pgmap v319: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: cluster 2024-09-10T18:40:47.327656+0000 mon.a (mon.0) 1802 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: audit 2024-09-10T18:40:47.338863+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-25", "mode": "writeback"}]': finished 2024-09-10T18:40:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: audit 2024-09-10T18:40:47.339010+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: audit 2024-09-10T18:40:47.339190+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: cluster 2024-09-10T18:40:47.339476+0000 mon.a (mon.0) 1806 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:40:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: audit 2024-09-10T18:40:47.341140+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.100:0/83021208' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: audit 2024-09-10T18:40:47.343397+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]: dispatch 2024-09-10T18:40:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: audit 2024-09-10T18:40:47.364772+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: audit 2024-09-10T18:40:47.381803+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:48 smithi100 bash[31607]: cluster 2024-09-10T18:40:47.870351+0000 mgr.y (mgr.24512) 271 : cluster [DBG] pgmap v319: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:40:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.332875+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.345177+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.345518+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: cluster 2024-09-10T18:40:48.345985+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.351855+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.352264+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.100:0/182988730' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.377773+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.378106+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.378933+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.379224+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.379677+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:48.379931+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: cluster 2024-09-10T18:40:49.335598+0000 mon.a (mon.0) 1817 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:49.345786+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]': finished 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:49.346009+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.100:0/182988730' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:49.346455+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:49.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: cluster 2024-09-10T18:40:49.346660+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:40:49.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:49.347814+0000 mon.c (mon.2) 303 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:49 smithi151 bash[22565]: audit 2024-09-10T18:40:49.349759+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.332875+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.345177+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.345518+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: cluster 2024-09-10T18:40:48.345985+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.351855+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.352264+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.100:0/182988730' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.377773+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.378106+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.378933+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.379224+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.379677+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:48.379931+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: cluster 2024-09-10T18:40:49.335598+0000 mon.a (mon.0) 1817 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:49.345786+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]': finished 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:49.346009+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.100:0/182988730' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.332875+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.345177+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-72593-34"}]': finished 2024-09-10T18:40:49.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.345518+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: cluster 2024-09-10T18:40:48.345985+0000 mon.a (mon.0) 1811 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.351855+0000 mon.a (mon.0) 1812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.352264+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.100:0/182988730' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.377773+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.378106+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.378933+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.379224+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.379677+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:48.379931+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: cluster 2024-09-10T18:40:49.335598+0000 mon.a (mon.0) 1817 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:49.345786+0000 mon.a (mon.0) 1818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-25"}]': finished 2024-09-10T18:40:49.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:49.346009+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? 172.21.15.100:0/182988730' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-72601-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:49.346455+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: cluster 2024-09-10T18:40:49.346660+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:49.347814+0000 mon.c (mon.2) 303 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[18597]: audit 2024-09-10T18:40:49.349759+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:49.346455+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-72593-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: cluster 2024-09-10T18:40:49.346660+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:49.347814+0000 mon.c (mon.2) 303 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:49.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:49 smithi100 bash[31607]: audit 2024-09-10T18:40:49.349759+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:51.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:50 smithi100 bash[18597]: cluster 2024-09-10T18:40:49.871266+0000 mgr.y (mgr.24512) 272 : cluster [DBG] pgmap v322: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:51.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:50 smithi100 bash[31607]: cluster 2024-09-10T18:40:49.871266+0000 mgr.y (mgr.24512) 272 : cluster [DBG] pgmap v322: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:51.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:50 smithi151 bash[22565]: cluster 2024-09-10T18:40:49.871266+0000 mgr.y (mgr.24512) 272 : cluster [DBG] pgmap v322: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:51.780 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:51] "GET /metrics HTTP/1.1" 200 233335 "" "Prometheus/2.43.0" 2024-09-10T18:40:52.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[18597]: cluster 2024-09-10T18:40:50.774641+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:40:52.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[18597]: audit 2024-09-10T18:40:51.343200+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[18597]: audit 2024-09-10T18:40:51.351816+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[18597]: cluster 2024-09-10T18:40:51.352026+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[18597]: audit 2024-09-10T18:40:51.367050+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[31607]: cluster 2024-09-10T18:40:50.774641+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[31607]: audit 2024-09-10T18:40:51.343200+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[31607]: audit 2024-09-10T18:40:51.351816+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[31607]: cluster 2024-09-10T18:40:51.352026+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:40:52.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:51 smithi100 bash[31607]: audit 2024-09-10T18:40:51.367050+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:52.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:51 smithi151 bash[22565]: cluster 2024-09-10T18:40:50.774641+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:40:52.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:51 smithi151 bash[22565]: audit 2024-09-10T18:40:51.343200+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:52.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:51 smithi151 bash[22565]: audit 2024-09-10T18:40:51.351816+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-72593-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:52.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:51 smithi151 bash[22565]: cluster 2024-09-10T18:40:51.352026+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:40:52.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:51 smithi151 bash[22565]: audit 2024-09-10T18:40:51.367050+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:52.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:40:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:40:53.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:52 smithi100 bash[31607]: cluster 2024-09-10T18:40:51.872367+0000 mgr.y (mgr.24512) 273 : cluster [DBG] pgmap v325: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:40:53.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:52 smithi100 bash[31607]: audit 2024-09-10T18:40:52.356197+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:53.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:52 smithi100 bash[31607]: cluster 2024-09-10T18:40:52.356379+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:40:53.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:52 smithi100 bash[18597]: cluster 2024-09-10T18:40:51.872367+0000 mgr.y (mgr.24512) 273 : cluster [DBG] pgmap v325: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:40:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:52 smithi100 bash[18597]: audit 2024-09-10T18:40:52.356197+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:53.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:52 smithi100 bash[18597]: cluster 2024-09-10T18:40:52.356379+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:40:53.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:52 smithi151 bash[22565]: cluster 2024-09-10T18:40:51.872367+0000 mgr.y (mgr.24512) 273 : cluster [DBG] pgmap v325: 332 pgs: 40 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:40:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:52 smithi151 bash[22565]: audit 2024-09-10T18:40:52.356197+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:52 smithi151 bash[22565]: cluster 2024-09-10T18:40:52.356379+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:40:54.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[31607]: audit 2024-09-10T18:40:52.207526+0000 mgr.y (mgr.24512) 274 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[31607]: cluster 2024-09-10T18:40:52.780160+0000 mon.a (mon.0) 1829 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[31607]: cluster 2024-09-10T18:40:53.362252+0000 mon.a (mon.0) 1830 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[31607]: audit 2024-09-10T18:40:53.364190+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[31607]: audit 2024-09-10T18:40:53.365040+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[31607]: audit 2024-09-10T18:40:53.375696+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[31607]: audit 2024-09-10T18:40:53.392851+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[18597]: audit 2024-09-10T18:40:52.207526+0000 mgr.y (mgr.24512) 274 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[18597]: cluster 2024-09-10T18:40:52.780160+0000 mon.a (mon.0) 1829 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:54.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[18597]: cluster 2024-09-10T18:40:53.362252+0000 mon.a (mon.0) 1830 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:40:54.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[18597]: audit 2024-09-10T18:40:53.364190+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:54.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[18597]: audit 2024-09-10T18:40:53.365040+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:54.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[18597]: audit 2024-09-10T18:40:53.375696+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:54.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:53 smithi100 bash[18597]: audit 2024-09-10T18:40:53.392851+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:53 smithi151 bash[22565]: audit 2024-09-10T18:40:52.207526+0000 mgr.y (mgr.24512) 274 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:53 smithi151 bash[22565]: cluster 2024-09-10T18:40:52.780160+0000 mon.a (mon.0) 1829 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:53 smithi151 bash[22565]: cluster 2024-09-10T18:40:53.362252+0000 mon.a (mon.0) 1830 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:53 smithi151 bash[22565]: audit 2024-09-10T18:40:53.364190+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:53 smithi151 bash[22565]: audit 2024-09-10T18:40:53.365040+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:53 smithi151 bash[22565]: audit 2024-09-10T18:40:53.375696+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:53 smithi151 bash[22565]: audit 2024-09-10T18:40:53.392851+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:40:54.834 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:40:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: cluster 2024-09-10T18:40:53.873254+0000 mgr.y (mgr.24512) 275 : cluster [DBG] pgmap v328: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.333111+0000 mon.a (mon.0) 1833 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.335748+0000 mon.c (mon.2) 305 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.347644+0000 mon.b (mon.1) 149 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.361433+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.361645+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: cluster 2024-09-10T18:40:54.361744+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.363965+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:55.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.364556+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:54 smithi151 bash[22565]: audit 2024-09-10T18:40:54.365128+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: cluster 2024-09-10T18:40:53.873254+0000 mgr.y (mgr.24512) 275 : cluster [DBG] pgmap v328: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:40:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.333111+0000 mon.a (mon.0) 1833 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.335748+0000 mon.c (mon.2) 305 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.347644+0000 mon.b (mon.1) 149 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.361433+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.361645+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: cluster 2024-09-10T18:40:54.361744+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.363965+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.364556+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[18597]: audit 2024-09-10T18:40:54.365128+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: cluster 2024-09-10T18:40:53.873254+0000 mgr.y (mgr.24512) 275 : cluster [DBG] pgmap v328: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.333111+0000 mon.a (mon.0) 1833 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:40:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.335748+0000 mon.c (mon.2) 305 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:55.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.347644+0000 mon.b (mon.1) 149 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:55.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.361433+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:55.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.361645+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:40:55.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: cluster 2024-09-10T18:40:54.361744+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:40:55.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.363965+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.100:0/185170083' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:55.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.364556+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]: dispatch 2024-09-10T18:40:55.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:54 smithi100 bash[31607]: audit 2024-09-10T18:40:54.365128+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.351126+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]: dispatch 2024-09-10T18:40:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.365605+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.365716+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]': finished 2024-09-10T18:40:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: cluster 2024-09-10T18:40:55.365834+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:40:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.368380+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? 172.21.15.100:0/3656474336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-72601-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.369466+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.390687+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.391122+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.391914+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.392381+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.392938+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: audit 2024-09-10T18:40:55.393257+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:56 smithi151 bash[22565]: cluster 2024-09-10T18:40:55.874168+0000 mgr.y (mgr.24512) 276 : cluster [DBG] pgmap v331: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:40:56.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.351126+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.365605+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.365716+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]': finished 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: cluster 2024-09-10T18:40:55.365834+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.368380+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? 172.21.15.100:0/3656474336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-72601-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.369466+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.390687+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.391122+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.391914+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.392381+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.392938+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: audit 2024-09-10T18:40:55.393257+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[18597]: cluster 2024-09-10T18:40:55.874168+0000 mgr.y (mgr.24512) 276 : cluster [DBG] pgmap v331: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.351126+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]: dispatch 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.365605+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-72593-35"}]': finished 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.365716+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-27"}]': finished 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: cluster 2024-09-10T18:40:55.365834+0000 mon.a (mon.0) 1841 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.368380+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? 172.21.15.100:0/3656474336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-72601-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.369466+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]: dispatch 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.390687+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.391122+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.391914+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.392381+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:56.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.392938+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: audit 2024-09-10T18:40:55.393257+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:56 smithi100 bash[31607]: cluster 2024-09-10T18:40:55.874168+0000 mgr.y (mgr.24512) 276 : cluster [DBG] pgmap v331: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:57 smithi151 bash[22565]: cluster 2024-09-10T18:40:56.359716+0000 mon.a (mon.0) 1847 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:57 smithi151 bash[22565]: audit 2024-09-10T18:40:56.377104+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? 172.21.15.100:0/3656474336' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-72601-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:57 smithi151 bash[22565]: audit 2024-09-10T18:40:56.377252+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]': finished 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:57 smithi151 bash[22565]: audit 2024-09-10T18:40:56.377459+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:57 smithi151 bash[22565]: cluster 2024-09-10T18:40:56.377529+0000 mon.a (mon.0) 1851 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:57 smithi151 bash[22565]: audit 2024-09-10T18:40:56.379729+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:57 smithi151 bash[22565]: audit 2024-09-10T18:40:56.385267+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:58.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[31607]: cluster 2024-09-10T18:40:56.359716+0000 mon.a (mon.0) 1847 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:58.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[31607]: audit 2024-09-10T18:40:56.377104+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? 172.21.15.100:0/3656474336' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-72601-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[31607]: audit 2024-09-10T18:40:56.377252+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]': finished 2024-09-10T18:40:58.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[31607]: audit 2024-09-10T18:40:56.377459+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[31607]: cluster 2024-09-10T18:40:56.377529+0000 mon.a (mon.0) 1851 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[31607]: audit 2024-09-10T18:40:56.379729+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[31607]: audit 2024-09-10T18:40:56.385267+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[18597]: cluster 2024-09-10T18:40:56.359716+0000 mon.a (mon.0) 1847 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[18597]: audit 2024-09-10T18:40:56.377104+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? 172.21.15.100:0/3656474336' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-72601-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[18597]: audit 2024-09-10T18:40:56.377252+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-27", "mode": "writeback"}]': finished 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[18597]: audit 2024-09-10T18:40:56.377459+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[18597]: cluster 2024-09-10T18:40:56.377529+0000 mon.a (mon.0) 1851 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[18597]: audit 2024-09-10T18:40:56.379729+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:57 smithi100 bash[18597]: audit 2024-09-10T18:40:56.385267+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: cluster 2024-09-10T18:40:57.814321+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: audit 2024-09-10T18:40:57.821483+0000 mon.b (mon.1) 151 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: audit 2024-09-10T18:40:57.839239+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: cluster 2024-09-10T18:40:57.874880+0000 mgr.y (mgr.24512) 277 : cluster [DBG] pgmap v334: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: audit 2024-09-10T18:40:58.370897+0000 mon.b (mon.1) 152 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: audit 2024-09-10T18:40:58.385175+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:40:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: audit 2024-09-10T18:40:58.385562+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: cluster 2024-09-10T18:40:58.385927+0000 mon.a (mon.0) 1857 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:40:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: audit 2024-09-10T18:40:58.388327+0000 mon.a (mon.0) 1858 : audit [INF] from='client.? 172.21.15.100:0/1211755641' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-72601-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:58 smithi151 bash[22565]: audit 2024-09-10T18:40:58.388729+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: cluster 2024-09-10T18:40:57.814321+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: audit 2024-09-10T18:40:57.821483+0000 mon.b (mon.1) 151 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: audit 2024-09-10T18:40:57.839239+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: cluster 2024-09-10T18:40:57.874880+0000 mgr.y (mgr.24512) 277 : cluster [DBG] pgmap v334: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: audit 2024-09-10T18:40:58.370897+0000 mon.b (mon.1) 152 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: audit 2024-09-10T18:40:58.385175+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: audit 2024-09-10T18:40:58.385562+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: cluster 2024-09-10T18:40:58.385927+0000 mon.a (mon.0) 1857 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: audit 2024-09-10T18:40:58.388327+0000 mon.a (mon.0) 1858 : audit [INF] from='client.? 172.21.15.100:0/1211755641' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-72601-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[18597]: audit 2024-09-10T18:40:58.388729+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: cluster 2024-09-10T18:40:57.814321+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: audit 2024-09-10T18:40:57.821483+0000 mon.b (mon.1) 151 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: audit 2024-09-10T18:40:57.839239+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: cluster 2024-09-10T18:40:57.874880+0000 mgr.y (mgr.24512) 277 : cluster [DBG] pgmap v334: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: audit 2024-09-10T18:40:58.370897+0000 mon.b (mon.1) 152 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: audit 2024-09-10T18:40:58.385175+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-72593-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: audit 2024-09-10T18:40:58.385562+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:40:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: cluster 2024-09-10T18:40:58.385927+0000 mon.a (mon.0) 1857 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:40:59.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: audit 2024-09-10T18:40:58.388327+0000 mon.a (mon.0) 1858 : audit [INF] from='client.? 172.21.15.100:0/1211755641' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-72601-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:58 smithi100 bash[31607]: audit 2024-09-10T18:40:58.388729+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]: dispatch 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:59 smithi151 bash[22565]: cluster 2024-09-10T18:40:58.816778+0000 mon.a (mon.0) 1860 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:59 smithi151 bash[22565]: cluster 2024-09-10T18:40:59.376411+0000 mon.a (mon.0) 1861 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:59 smithi151 bash[22565]: audit 2024-09-10T18:40:59.385877+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? 172.21.15.100:0/1211755641' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-72601-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:59 smithi151 bash[22565]: audit 2024-09-10T18:40:59.386050+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]': finished 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:40:59 smithi151 bash[22565]: cluster 2024-09-10T18:40:59.386497+0000 mon.a (mon.0) 1864 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:41:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[31607]: cluster 2024-09-10T18:40:58.816778+0000 mon.a (mon.0) 1860 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[31607]: cluster 2024-09-10T18:40:59.376411+0000 mon.a (mon.0) 1861 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[31607]: audit 2024-09-10T18:40:59.385877+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? 172.21.15.100:0/1211755641' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-72601-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[31607]: audit 2024-09-10T18:40:59.386050+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]': finished 2024-09-10T18:41:00.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[31607]: cluster 2024-09-10T18:40:59.386497+0000 mon.a (mon.0) 1864 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:41:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[18597]: cluster 2024-09-10T18:40:58.816778+0000 mon.a (mon.0) 1860 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[18597]: cluster 2024-09-10T18:40:59.376411+0000 mon.a (mon.0) 1861 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[18597]: audit 2024-09-10T18:40:59.385877+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? 172.21.15.100:0/1211755641' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-72601-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[18597]: audit 2024-09-10T18:40:59.386050+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-27"}]': finished 2024-09-10T18:41:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:40:59 smithi100 bash[18597]: cluster 2024-09-10T18:40:59.386497+0000 mon.a (mon.0) 1864 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:41:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: cluster 2024-09-10T18:40:59.875772+0000 mgr.y (mgr.24512) 278 : cluster [DBG] pgmap v337: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: cluster 2024-09-10T18:41:00.388058+0000 mon.a (mon.0) 1865 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:41:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.392934+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.418858+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.822118+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.822354+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.823424+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.824012+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.825086+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:01.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:00 smithi151 bash[22565]: audit 2024-09-10T18:41:00.825652+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: cluster 2024-09-10T18:40:59.875772+0000 mgr.y (mgr.24512) 278 : cluster [DBG] pgmap v337: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: cluster 2024-09-10T18:41:00.388058+0000 mon.a (mon.0) 1865 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.392934+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.418858+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.822118+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.822354+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.823424+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.824012+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.825086+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[18597]: audit 2024-09-10T18:41:00.825652+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: cluster 2024-09-10T18:40:59.875772+0000 mgr.y (mgr.24512) 278 : cluster [DBG] pgmap v337: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: cluster 2024-09-10T18:41:00.388058+0000 mon.a (mon.0) 1865 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.392934+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.418858+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.822118+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.822354+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.823424+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.824012+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:01.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.825086+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:01.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:00 smithi100 bash[31607]: audit 2024-09-10T18:41:00.825652+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:41:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:01] "GET /metrics HTTP/1.1" 200 230907 "" "Prometheus/2.43.0" 2024-09-10T18:41:02.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:41:02.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.380113+0000 mon.b (mon.1) 153 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.393142+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.393359+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: cluster 2024-09-10T18:41:01.393906+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.396822+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.397277+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.398928+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.399732+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: audit 2024-09-10T18:41:01.400066+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:02 smithi151 bash[22565]: cluster 2024-09-10T18:41:01.876614+0000 mgr.y (mgr.24512) 279 : cluster [DBG] pgmap v340: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.380113+0000 mon.b (mon.1) 153 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.393142+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:41:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.393359+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: cluster 2024-09-10T18:41:01.393906+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:41:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.396822+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.397277+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.398928+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.399732+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: audit 2024-09-10T18:41:01.400066+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[18597]: cluster 2024-09-10T18:41:01.876614+0000 mgr.y (mgr.24512) 279 : cluster [DBG] pgmap v340: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.380113+0000 mon.b (mon.1) 153 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.393142+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.393359+0000 mon.a (mon.0) 1871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: cluster 2024-09-10T18:41:01.393906+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.396822+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.397277+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.100:0/842307170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.398928+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:02.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.399732+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]: dispatch 2024-09-10T18:41:02.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: audit 2024-09-10T18:41:01.400066+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:02 smithi100 bash[31607]: cluster 2024-09-10T18:41:01.876614+0000 mgr.y (mgr.24512) 279 : cluster [DBG] pgmap v340: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.219192+0000 mgr.y (mgr.24512) 280 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.815589+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.815773+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: cluster 2024-09-10T18:41:02.816275+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.839045+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.839555+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.840666+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.841067+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.841784+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:03.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:03 smithi151 bash[22565]: audit 2024-09-10T18:41:02.842160+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:03.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.219192+0000 mgr.y (mgr.24512) 280 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:03.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.815589+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:41:03.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.815773+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: cluster 2024-09-10T18:41:02.816275+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.839045+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.839555+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.840666+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.841067+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.841784+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[18597]: audit 2024-09-10T18:41:02.842160+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.219192+0000 mgr.y (mgr.24512) 280 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.815589+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-72593-36"}]': finished 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.815773+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: cluster 2024-09-10T18:41:02.816275+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:41:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.839045+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.839555+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.840666+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.841067+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.841784+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:03 smithi100 bash[31607]: audit 2024-09-10T18:41:02.842160+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:04.846 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:41:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:05.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:04 smithi151 bash[22565]: audit 2024-09-10T18:41:03.409292+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:05.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:04 smithi151 bash[22565]: audit 2024-09-10T18:41:03.409429+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:05.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:04 smithi151 bash[22565]: cluster 2024-09-10T18:41:03.409593+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:41:05.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:04 smithi151 bash[22565]: audit 2024-09-10T18:41:03.412663+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:05.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:04 smithi151 bash[22565]: audit 2024-09-10T18:41:03.420902+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:05.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:04 smithi151 bash[22565]: cluster 2024-09-10T18:41:03.877468+0000 mgr.y (mgr.24512) 281 : cluster [DBG] pgmap v343: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[18597]: audit 2024-09-10T18:41:03.409292+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[18597]: audit 2024-09-10T18:41:03.409429+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[18597]: cluster 2024-09-10T18:41:03.409593+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[18597]: audit 2024-09-10T18:41:03.412663+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[18597]: audit 2024-09-10T18:41:03.420902+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[18597]: cluster 2024-09-10T18:41:03.877468+0000 mgr.y (mgr.24512) 281 : cluster [DBG] pgmap v343: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[31607]: audit 2024-09-10T18:41:03.409292+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[31607]: audit 2024-09-10T18:41:03.409429+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-72593-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[31607]: cluster 2024-09-10T18:41:03.409593+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[31607]: audit 2024-09-10T18:41:03.412663+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[31607]: audit 2024-09-10T18:41:03.420902+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:05.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:04 smithi100 bash[31607]: cluster 2024-09-10T18:41:03.877468+0000 mgr.y (mgr.24512) 281 : cluster [DBG] pgmap v343: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:04.839174+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: cluster 2024-09-10T18:41:04.843196+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:41:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:04.861554+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:05.316735+0000 mon.b (mon.1) 155 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: cluster 2024-09-10T18:41:05.318445+0000 mon.a (mon.0) 1888 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:05.331749+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:05.332100+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: cluster 2024-09-10T18:41:05.332255+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:41:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:05.334933+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:05.336090+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:05 smithi151 bash[22565]: audit 2024-09-10T18:41:05.337567+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:06.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:04.839174+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:06.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: cluster 2024-09-10T18:41:04.843196+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:04.861554+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:05.316735+0000 mon.b (mon.1) 155 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: cluster 2024-09-10T18:41:05.318445+0000 mon.a (mon.0) 1888 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:05.331749+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:05.332100+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: cluster 2024-09-10T18:41:05.332255+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:05.334933+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:05.336090+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[18597]: audit 2024-09-10T18:41:05.337567+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:04.839174+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: cluster 2024-09-10T18:41:04.843196+0000 mon.a (mon.0) 1886 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:04.861554+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:05.316735+0000 mon.b (mon.1) 155 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: cluster 2024-09-10T18:41:05.318445+0000 mon.a (mon.0) 1888 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:05.331749+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-72593-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:05.332100+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: cluster 2024-09-10T18:41:05.332255+0000 mon.a (mon.0) 1891 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:05.334933+0000 mon.a (mon.0) 1892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:05.336090+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:05 smithi100 bash[31607]: audit 2024-09-10T18:41:05.337567+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:06.856 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: cluster 2024-09-10T18:41:05.878171+0000 mgr.y (mgr.24512) 282 : cluster [DBG] pgmap v346: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:41:06.857 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: audit 2024-09-10T18:41:06.318963+0000 mon.b (mon.1) 156 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]: dispatch 2024-09-10T18:41:06.857 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: audit 2024-09-10T18:41:06.334181+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]': finished 2024-09-10T18:41:06.857 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: audit 2024-09-10T18:41:06.334300+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:06.857 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: cluster 2024-09-10T18:41:06.334482+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:41:06.857 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: audit 2024-09-10T18:41:06.335634+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:06.857 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: audit 2024-09-10T18:41:06.337047+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]: dispatch 2024-09-10T18:41:06.857 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:06 smithi151 bash[22565]: audit 2024-09-10T18:41:06.337406+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: cluster 2024-09-10T18:41:05.878171+0000 mgr.y (mgr.24512) 282 : cluster [DBG] pgmap v346: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:41:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: audit 2024-09-10T18:41:06.318963+0000 mon.b (mon.1) 156 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]: dispatch 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: audit 2024-09-10T18:41:06.334181+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]': finished 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: audit 2024-09-10T18:41:06.334300+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: cluster 2024-09-10T18:41:06.334482+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: audit 2024-09-10T18:41:06.335634+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: audit 2024-09-10T18:41:06.337047+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]: dispatch 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[18597]: audit 2024-09-10T18:41:06.337406+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: cluster 2024-09-10T18:41:05.878171+0000 mgr.y (mgr.24512) 282 : cluster [DBG] pgmap v346: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: audit 2024-09-10T18:41:06.318963+0000 mon.b (mon.1) 156 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]: dispatch 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: audit 2024-09-10T18:41:06.334181+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-29"}]': finished 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: audit 2024-09-10T18:41:06.334300+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:07.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: cluster 2024-09-10T18:41:06.334482+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:41:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: audit 2024-09-10T18:41:06.335634+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.100:0/186911881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: audit 2024-09-10T18:41:06.337047+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]: dispatch 2024-09-10T18:41:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:06 smithi100 bash[31607]: audit 2024-09-10T18:41:06.337406+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]: dispatch 2024-09-10T18:41:08.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: cluster 2024-09-10T18:41:07.326245+0000 mon.a (mon.0) 1899 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:08.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.338694+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]': finished 2024-09-10T18:41:08.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.338927+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:08.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: cluster 2024-09-10T18:41:07.339095+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:41:08.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.342813+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:08.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.343981+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:08.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.371245+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.371966+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.372662+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.372938+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.373419+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:08.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:07 smithi151 bash[22565]: audit 2024-09-10T18:41:07.373678+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:08.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: cluster 2024-09-10T18:41:07.326245+0000 mon.a (mon.0) 1899 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:08.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.338694+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]': finished 2024-09-10T18:41:08.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.338927+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: cluster 2024-09-10T18:41:07.339095+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.342813+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.343981+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.371245+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.371966+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.372662+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.372938+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.373419+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[18597]: audit 2024-09-10T18:41:07.373678+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:08.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: cluster 2024-09-10T18:41:07.326245+0000 mon.a (mon.0) 1899 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.338694+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-29", "mode": "writeback"}]': finished 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.338927+0000 mon.a (mon.0) 1901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-46"}]': finished 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: cluster 2024-09-10T18:41:07.339095+0000 mon.a (mon.0) 1902 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.342813+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.343981+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.371245+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.371966+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.372662+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.372938+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.373419+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:07 smithi100 bash[31607]: audit 2024-09-10T18:41:07.373678+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: cluster 2024-09-10T18:41:07.878652+0000 mgr.y (mgr.24512) 283 : cluster [DBG] pgmap v349: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: audit 2024-09-10T18:41:08.352987+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: audit 2024-09-10T18:41:08.353173+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: cluster 2024-09-10T18:41:08.353341+0000 mon.a (mon.0) 1909 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: audit 2024-09-10T18:41:08.356371+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: audit 2024-09-10T18:41:08.356658+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: audit 2024-09-10T18:41:08.357016+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:09.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:08 smithi151 bash[22565]: audit 2024-09-10T18:41:08.357711+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: cluster 2024-09-10T18:41:07.878652+0000 mgr.y (mgr.24512) 283 : cluster [DBG] pgmap v349: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:41:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: audit 2024-09-10T18:41:08.352987+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: audit 2024-09-10T18:41:08.353173+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: cluster 2024-09-10T18:41:08.353341+0000 mon.a (mon.0) 1909 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:41:09.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: audit 2024-09-10T18:41:08.356371+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: audit 2024-09-10T18:41:08.356658+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: audit 2024-09-10T18:41:08.357016+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[18597]: audit 2024-09-10T18:41:08.357711+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: cluster 2024-09-10T18:41:07.878652+0000 mgr.y (mgr.24512) 283 : cluster [DBG] pgmap v349: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: audit 2024-09-10T18:41:08.352987+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: audit 2024-09-10T18:41:08.353173+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-72601-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: cluster 2024-09-10T18:41:08.353341+0000 mon.a (mon.0) 1909 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: audit 2024-09-10T18:41:08.356371+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.100:0/791181408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: audit 2024-09-10T18:41:08.356658+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: audit 2024-09-10T18:41:08.357016+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]: dispatch 2024-09-10T18:41:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:08 smithi100 bash[31607]: audit 2024-09-10T18:41:08.357711+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:10.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.332995+0000 mon.a (mon.0) 1912 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:10.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.335726+0000 mon.c (mon.2) 329 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:10.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.358077+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:10.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.382493+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.412448+0000 mon.c (mon.2) 331 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: cluster 2024-09-10T18:41:09.798259+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.799661+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.800681+0000 mon.a (mon.0) 1916 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.800870+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.801816+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.802828+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: audit 2024-09-10T18:41:09.803532+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: cluster 2024-09-10T18:41:09.879390+0000 mgr.y (mgr.24512) 284 : cluster [DBG] pgmap v352: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:10.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:10 smithi151 bash[22565]: cluster 2024-09-10T18:41:10.321470+0000 mon.a (mon.0) 1919 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.332995+0000 mon.a (mon.0) 1912 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.335726+0000 mon.c (mon.2) 329 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.358077+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.382493+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.412448+0000 mon.c (mon.2) 331 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: cluster 2024-09-10T18:41:09.798259+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.799661+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.800681+0000 mon.a (mon.0) 1916 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.800870+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.801816+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.802828+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: audit 2024-09-10T18:41:09.803532+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: cluster 2024-09-10T18:41:09.879390+0000 mgr.y (mgr.24512) 284 : cluster [DBG] pgmap v352: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[18597]: cluster 2024-09-10T18:41:10.321470+0000 mon.a (mon.0) 1919 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.332995+0000 mon.a (mon.0) 1912 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.335726+0000 mon.c (mon.2) 329 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.358077+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-72593-37"}]': finished 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.382493+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.412448+0000 mon.c (mon.2) 331 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: cluster 2024-09-10T18:41:09.798259+0000 mon.a (mon.0) 1914 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.799661+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.800681+0000 mon.a (mon.0) 1916 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.800870+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.801816+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.802828+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: audit 2024-09-10T18:41:09.803532+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: cluster 2024-09-10T18:41:09.879390+0000 mgr.y (mgr.24512) 284 : cluster [DBG] pgmap v352: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:10 smithi100 bash[31607]: cluster 2024-09-10T18:41:10.321470+0000 mon.a (mon.0) 1919 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:11] "GET /metrics HTTP/1.1" 200 228428 "" "Prometheus/2.43.0" 2024-09-10T18:41:12.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:11 smithi151 bash[22565]: audit 2024-09-10T18:41:10.361889+0000 mon.a (mon.0) 1920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:12.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:11 smithi151 bash[22565]: audit 2024-09-10T18:41:10.362088+0000 mon.a (mon.0) 1921 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]': finished 2024-09-10T18:41:12.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:11 smithi151 bash[22565]: audit 2024-09-10T18:41:10.362320+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:12.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:11 smithi151 bash[22565]: cluster 2024-09-10T18:41:10.362469+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:41:12.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:11 smithi151 bash[22565]: audit 2024-09-10T18:41:10.368556+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:12.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:11 smithi151 bash[22565]: audit 2024-09-10T18:41:10.369881+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[18597]: audit 2024-09-10T18:41:10.361889+0000 mon.a (mon.0) 1920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[18597]: audit 2024-09-10T18:41:10.362088+0000 mon.a (mon.0) 1921 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]': finished 2024-09-10T18:41:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[18597]: audit 2024-09-10T18:41:10.362320+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[18597]: cluster 2024-09-10T18:41:10.362469+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[18597]: audit 2024-09-10T18:41:10.368556+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[18597]: audit 2024-09-10T18:41:10.369881+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[31607]: audit 2024-09-10T18:41:10.361889+0000 mon.a (mon.0) 1920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-72601-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[31607]: audit 2024-09-10T18:41:10.362088+0000 mon.a (mon.0) 1921 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "172.7", "id": [1, 7]}]': finished 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[31607]: audit 2024-09-10T18:41:10.362320+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-72593-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[31607]: cluster 2024-09-10T18:41:10.362469+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[31607]: audit 2024-09-10T18:41:10.368556+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:11 smithi100 bash[31607]: audit 2024-09-10T18:41:10.369881+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:12.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:41:13.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: cluster 2024-09-10T18:41:11.796788+0000 mon.a (mon.0) 1925 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:41:13.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:11.826296+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:13.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:11.843789+0000 mon.a (mon.0) 1926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: cluster 2024-09-10T18:41:11.880202+0000 mgr.y (mgr.24512) 285 : cluster [DBG] pgmap v355: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:12.356510+0000 mon.b (mon.1) 158 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:12.369627+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:12.369959+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: cluster 2024-09-10T18:41:12.370123+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:12.374089+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:12.374751+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:13.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:12 smithi151 bash[22565]: audit 2024-09-10T18:41:12.375221+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:13.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: cluster 2024-09-10T18:41:11.796788+0000 mon.a (mon.0) 1925 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:41:13.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:11.826296+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:11.843789+0000 mon.a (mon.0) 1926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: cluster 2024-09-10T18:41:11.880202+0000 mgr.y (mgr.24512) 285 : cluster [DBG] pgmap v355: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:12.356510+0000 mon.b (mon.1) 158 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:12.369627+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:12.369959+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: cluster 2024-09-10T18:41:12.370123+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:12.374089+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:12.374751+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[18597]: audit 2024-09-10T18:41:12.375221+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:13.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: cluster 2024-09-10T18:41:11.796788+0000 mon.a (mon.0) 1925 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:11.826296+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:11.843789+0000 mon.a (mon.0) 1926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: cluster 2024-09-10T18:41:11.880202+0000 mgr.y (mgr.24512) 285 : cluster [DBG] pgmap v355: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:12.356510+0000 mon.b (mon.1) 158 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:12.369627+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-72593-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:12.369959+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: cluster 2024-09-10T18:41:12.370123+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:12.374089+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]: dispatch 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:12.374751+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:12 smithi100 bash[31607]: audit 2024-09-10T18:41:12.375221+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:14.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:13 smithi151 bash[22565]: audit 2024-09-10T18:41:12.223848+0000 mgr.y (mgr.24512) 286 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:14.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:13 smithi151 bash[22565]: cluster 2024-09-10T18:41:13.363488+0000 mon.a (mon.0) 1932 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:13 smithi151 bash[22565]: audit 2024-09-10T18:41:13.372716+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]': finished 2024-09-10T18:41:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:13 smithi151 bash[22565]: audit 2024-09-10T18:41:13.372913+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:13 smithi151 bash[22565]: audit 2024-09-10T18:41:13.374297+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:13 smithi151 bash[22565]: cluster 2024-09-10T18:41:13.375528+0000 mon.a (mon.0) 1935 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:41:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:13 smithi151 bash[22565]: audit 2024-09-10T18:41:13.376812+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[18597]: audit 2024-09-10T18:41:12.223848+0000 mgr.y (mgr.24512) 286 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[18597]: cluster 2024-09-10T18:41:13.363488+0000 mon.a (mon.0) 1932 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[18597]: audit 2024-09-10T18:41:13.372716+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]': finished 2024-09-10T18:41:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[18597]: audit 2024-09-10T18:41:13.372913+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[18597]: audit 2024-09-10T18:41:13.374297+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[18597]: cluster 2024-09-10T18:41:13.375528+0000 mon.a (mon.0) 1935 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[18597]: audit 2024-09-10T18:41:13.376812+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[31607]: audit 2024-09-10T18:41:12.223848+0000 mgr.y (mgr.24512) 286 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[31607]: cluster 2024-09-10T18:41:13.363488+0000 mon.a (mon.0) 1932 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[31607]: audit 2024-09-10T18:41:13.372716+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-29"}]': finished 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[31607]: audit 2024-09-10T18:41:13.372913+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[31607]: audit 2024-09-10T18:41:13.374297+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.100:0/609921701' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[31607]: cluster 2024-09-10T18:41:13.375528+0000 mon.a (mon.0) 1935 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:41:14.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:13 smithi100 bash[31607]: audit 2024-09-10T18:41:13.376812+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]: dispatch 2024-09-10T18:41:14.827 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:15.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: cluster 2024-09-10T18:41:13.881033+0000 mgr.y (mgr.24512) 287 : cluster [DBG] pgmap v358: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-10T18:41:15.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.378565+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:15.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: cluster 2024-09-10T18:41:14.378779+0000 mon.a (mon.0) 1938 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:41:15.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.382381+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:15.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.406293+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.811053+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:15.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.811658+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.812817+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.813450+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.814468+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:15.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:14 smithi151 bash[22565]: audit 2024-09-10T18:41:14.815097+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:15.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: cluster 2024-09-10T18:41:13.881033+0000 mgr.y (mgr.24512) 287 : cluster [DBG] pgmap v358: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-10T18:41:15.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.378565+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: cluster 2024-09-10T18:41:14.378779+0000 mon.a (mon.0) 1938 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.382381+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.406293+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.811053+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.811658+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.812817+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.813450+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.814468+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[18597]: audit 2024-09-10T18:41:14.815097+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: cluster 2024-09-10T18:41:13.881033+0000 mgr.y (mgr.24512) 287 : cluster [DBG] pgmap v358: 236 pgs: 8 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-10T18:41:15.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.378565+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-72601-47"}]': finished 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: cluster 2024-09-10T18:41:14.378779+0000 mon.a (mon.0) 1938 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.382381+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.406293+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.811053+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.811658+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.812817+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.813450+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.814468+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:15.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:14 smithi100 bash[31607]: audit 2024-09-10T18:41:14.815097+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:16.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.321594+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: cluster 2024-09-10T18:41:15.323311+0000 mon.a (mon.0) 1943 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:16.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.335826+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:16.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.335944+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:16.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: cluster 2024-09-10T18:41:15.336157+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:41:16.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.337208+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:16.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.337500+0000 mon.c (mon.2) 342 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:16.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.338273+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:16.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.339770+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:15 smithi151 bash[22565]: audit 2024-09-10T18:41:15.340041+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:16.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.321594+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: cluster 2024-09-10T18:41:15.323311+0000 mon.a (mon.0) 1943 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:16.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.335826+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.335944+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: cluster 2024-09-10T18:41:15.336157+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.337208+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.337500+0000 mon.c (mon.2) 342 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.338273+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.339770+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[18597]: audit 2024-09-10T18:41:15.340041+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.321594+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: cluster 2024-09-10T18:41:15.323311+0000 mon.a (mon.0) 1943 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.335826+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:16.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.335944+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-72601-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: cluster 2024-09-10T18:41:15.336157+0000 mon.a (mon.0) 1946 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:41:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.337208+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.337500+0000 mon.c (mon.2) 342 : audit [INF] from='client.? 172.21.15.100:0/62903895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.338273+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.339770+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:15 smithi100 bash[31607]: audit 2024-09-10T18:41:15.340041+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]: dispatch 2024-09-10T18:41:17.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: cluster 2024-09-10T18:41:15.881816+0000 mgr.y (mgr.24512) 288 : cluster [DBG] pgmap v361: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:17.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: audit 2024-09-10T18:41:16.754655+0000 mon.b (mon.1) 160 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:17.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: audit 2024-09-10T18:41:16.769567+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:17.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: audit 2024-09-10T18:41:16.769813+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:17.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: cluster 2024-09-10T18:41:16.770453+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:41:17.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: audit 2024-09-10T18:41:16.775357+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:17.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: audit 2024-09-10T18:41:16.793181+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:17.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: audit 2024-09-10T18:41:16.795591+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:17.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:16 smithi151 bash[22565]: audit 2024-09-10T18:41:16.796220+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-72593-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:17.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: cluster 2024-09-10T18:41:15.881816+0000 mgr.y (mgr.24512) 288 : cluster [DBG] pgmap v361: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: audit 2024-09-10T18:41:16.754655+0000 mon.b (mon.1) 160 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: audit 2024-09-10T18:41:16.769567+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: audit 2024-09-10T18:41:16.769813+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: cluster 2024-09-10T18:41:16.770453+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: audit 2024-09-10T18:41:16.775357+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: audit 2024-09-10T18:41:16.793181+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: audit 2024-09-10T18:41:16.795591+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[18597]: audit 2024-09-10T18:41:16.796220+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-72593-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: cluster 2024-09-10T18:41:15.881816+0000 mgr.y (mgr.24512) 288 : cluster [DBG] pgmap v361: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: audit 2024-09-10T18:41:16.754655+0000 mon.b (mon.1) 160 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: audit 2024-09-10T18:41:16.769567+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: audit 2024-09-10T18:41:16.769813+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-72593-38"}]': finished 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: cluster 2024-09-10T18:41:16.770453+0000 mon.a (mon.0) 1952 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: audit 2024-09-10T18:41:16.775357+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: audit 2024-09-10T18:41:16.793181+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: audit 2024-09-10T18:41:16.795591+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:17.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:16 smithi100 bash[31607]: audit 2024-09-10T18:41:16.796220+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-72593-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: audit 2024-09-10T18:41:17.330729+0000 mon.b (mon.1) 161 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: audit 2024-09-10T18:41:17.345716+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: audit 2024-09-10T18:41:17.345847+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: audit 2024-09-10T18:41:17.345953+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-72593-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:19.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: cluster 2024-09-10T18:41:17.346024+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:41:19.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: audit 2024-09-10T18:41:17.350584+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:19.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: audit 2024-09-10T18:41:17.351292+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-72593-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:19.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[18597]: cluster 2024-09-10T18:41:17.882651+0000 mgr.y (mgr.24512) 289 : cluster [DBG] pgmap v364: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:19.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: audit 2024-09-10T18:41:17.330729+0000 mon.b (mon.1) 161 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:19.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: audit 2024-09-10T18:41:17.345716+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:19.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: audit 2024-09-10T18:41:17.345847+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:19.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: audit 2024-09-10T18:41:17.345953+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-72593-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:19.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: cluster 2024-09-10T18:41:17.346024+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:41:19.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: audit 2024-09-10T18:41:17.350584+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:19.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: audit 2024-09-10T18:41:17.351292+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-72593-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:19.047 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:18 smithi100 bash[31607]: cluster 2024-09-10T18:41:17.882651+0000 mgr.y (mgr.24512) 289 : cluster [DBG] pgmap v364: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:19.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: audit 2024-09-10T18:41:17.330729+0000 mon.b (mon.1) 161 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:19.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: audit 2024-09-10T18:41:17.345716+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-72601-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:19.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: audit 2024-09-10T18:41:17.345847+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:19.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: audit 2024-09-10T18:41:17.345953+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-72593-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:19.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: cluster 2024-09-10T18:41:17.346024+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:41:19.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: audit 2024-09-10T18:41:17.350584+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:19.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: audit 2024-09-10T18:41:17.351292+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-72593-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:19.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:18 smithi151 bash[22565]: cluster 2024-09-10T18:41:17.882651+0000 mgr.y (mgr.24512) 289 : cluster [DBG] pgmap v364: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:18.333141+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:18.349115+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]': finished 2024-09-10T18:41:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: cluster 2024-09-10T18:41:18.768710+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:41:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:18.770568+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: cluster 2024-09-10T18:41:19.340722+0000 mon.a (mon.0) 1966 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:19.349804+0000 mon.a (mon.0) 1967 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-72593-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:19.350007+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]': finished 2024-09-10T18:41:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: cluster 2024-09-10T18:41:19.350220+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:19.354059+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:19.355217+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:19.473923+0000 mon.b (mon.1) 163 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[18597]: audit 2024-09-10T18:41:19.491725+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:18.333141+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:18.349115+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]': finished 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: cluster 2024-09-10T18:41:18.768710+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:18.770568+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: cluster 2024-09-10T18:41:19.340722+0000 mon.a (mon.0) 1966 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:19.349804+0000 mon.a (mon.0) 1967 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-72593-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:19.350007+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]': finished 2024-09-10T18:41:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: cluster 2024-09-10T18:41:19.350220+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:41:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:19.354059+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:19.355217+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:19.473923+0000 mon.b (mon.1) 163 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:19 smithi100 bash[31607]: audit 2024-09-10T18:41:19.491725+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:20.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:18.333141+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:18.349115+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-31"}]': finished 2024-09-10T18:41:20.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: cluster 2024-09-10T18:41:18.768710+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:18.770568+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: cluster 2024-09-10T18:41:19.340722+0000 mon.a (mon.0) 1966 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:19.349804+0000 mon.a (mon.0) 1967 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-72593-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:19.350007+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-31", "mode": "writeback"}]': finished 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: cluster 2024-09-10T18:41:19.350220+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:19.354059+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:19.355217+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:20.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:19.473923+0000 mon.b (mon.1) 163 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:20.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:19 smithi151 bash[22565]: audit 2024-09-10T18:41:19.491725+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:21.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: cluster 2024-09-10T18:41:19.883461+0000 mgr.y (mgr.24512) 290 : cluster [DBG] pgmap v367: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:21.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: audit 2024-09-10T18:41:20.323967+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: audit 2024-09-10T18:41:20.337391+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: audit 2024-09-10T18:41:20.338267+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: cluster 2024-09-10T18:41:20.338616+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: audit 2024-09-10T18:41:20.339224+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: audit 2024-09-10T18:41:20.340337+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[31607]: audit 2024-09-10T18:41:20.341725+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: cluster 2024-09-10T18:41:19.883461+0000 mgr.y (mgr.24512) 290 : cluster [DBG] pgmap v367: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: audit 2024-09-10T18:41:20.323967+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: audit 2024-09-10T18:41:20.337391+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: audit 2024-09-10T18:41:20.338267+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: cluster 2024-09-10T18:41:20.338616+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:41:21.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: audit 2024-09-10T18:41:20.339224+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:21.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: audit 2024-09-10T18:41:20.340337+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:21.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:20 smithi100 bash[18597]: audit 2024-09-10T18:41:20.341725+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:21.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: cluster 2024-09-10T18:41:19.883461+0000 mgr.y (mgr.24512) 290 : cluster [DBG] pgmap v367: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:21.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: audit 2024-09-10T18:41:20.323967+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:21.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: audit 2024-09-10T18:41:20.337391+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:21.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: audit 2024-09-10T18:41:20.338267+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:21.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: cluster 2024-09-10T18:41:20.338616+0000 mon.a (mon.0) 1974 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:41:21.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: audit 2024-09-10T18:41:20.339224+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.100:0/1134591609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:21.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: audit 2024-09-10T18:41:20.340337+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]: dispatch 2024-09-10T18:41:21.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:20 smithi151 bash[22565]: audit 2024-09-10T18:41:20.341725+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]: dispatch 2024-09-10T18:41:21.790 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:21] "GET /metrics HTTP/1.1" 200 228428 "" "Prometheus/2.43.0" 2024-09-10T18:41:22.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: cluster 2024-09-10T18:41:21.331253+0000 mon.a (mon.0) 1977 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.341442+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.341647+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]': finished 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: cluster 2024-09-10T18:41:21.341890+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.344130+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.352691+0000 mon.b (mon.1) 165 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.355208+0000 mon.b (mon.1) 166 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.356462+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.370305+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.372718+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[18597]: audit 2024-09-10T18:41:21.373945+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: cluster 2024-09-10T18:41:21.331253+0000 mon.a (mon.0) 1977 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.341442+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.341647+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]': finished 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: cluster 2024-09-10T18:41:21.341890+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.344130+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.352691+0000 mon.b (mon.1) 165 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.355208+0000 mon.b (mon.1) 166 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.356462+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.370305+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.372718+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:21 smithi100 bash[31607]: audit 2024-09-10T18:41:21.373945+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:22.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: cluster 2024-09-10T18:41:21.331253+0000 mon.a (mon.0) 1977 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:22.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.341442+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-72601-48"}]': finished 2024-09-10T18:41:22.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.341647+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-31"}]': finished 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: cluster 2024-09-10T18:41:21.341890+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.344130+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.352691+0000 mon.b (mon.1) 165 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.355208+0000 mon.b (mon.1) 166 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.356462+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.370305+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.372718+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:22.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:21 smithi151 bash[22565]: audit 2024-09-10T18:41:21.373945+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:22.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:41:23.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[22565]: cluster 2024-09-10T18:41:21.884272+0000 mgr.y (mgr.24512) 291 : cluster [DBG] pgmap v370: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:41:23.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[22565]: audit 2024-09-10T18:41:22.333047+0000 mon.b (mon.1) 168 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:23.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[22565]: audit 2024-09-10T18:41:22.348526+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:23.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[22565]: audit 2024-09-10T18:41:22.348684+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:23.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[22565]: cluster 2024-09-10T18:41:22.348880+0000 mon.a (mon.0) 1987 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:41:23.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[22565]: audit 2024-09-10T18:41:22.350263+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:23.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:22 smithi151 bash[22565]: audit 2024-09-10T18:41:22.351300+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:23.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[31607]: cluster 2024-09-10T18:41:21.884272+0000 mgr.y (mgr.24512) 291 : cluster [DBG] pgmap v370: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:41:23.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[31607]: audit 2024-09-10T18:41:22.333047+0000 mon.b (mon.1) 168 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[31607]: audit 2024-09-10T18:41:22.348526+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[31607]: audit 2024-09-10T18:41:22.348684+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[31607]: cluster 2024-09-10T18:41:22.348880+0000 mon.a (mon.0) 1987 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[31607]: audit 2024-09-10T18:41:22.350263+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[31607]: audit 2024-09-10T18:41:22.351300+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[18597]: cluster 2024-09-10T18:41:21.884272+0000 mgr.y (mgr.24512) 291 : cluster [DBG] pgmap v370: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[18597]: audit 2024-09-10T18:41:22.333047+0000 mon.b (mon.1) 168 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:23.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[18597]: audit 2024-09-10T18:41:22.348526+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:23.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[18597]: audit 2024-09-10T18:41:22.348684+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-72601-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:23.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[18597]: cluster 2024-09-10T18:41:22.348880+0000 mon.a (mon.0) 1987 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:41:23.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[18597]: audit 2024-09-10T18:41:22.350263+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]: dispatch 2024-09-10T18:41:23.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:22 smithi100 bash[18597]: audit 2024-09-10T18:41:22.351300+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:24.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:22.235464+0000 mgr.y (mgr.24512) 292 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: cluster 2024-09-10T18:41:22.801243+0000 mon.a (mon.0) 1990 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.349401+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.754767+0000 mon.b (mon.1) 169 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: cluster 2024-09-10T18:41:23.769746+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.773275+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.796714+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.797546+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.798760+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.799281+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.800156+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:24.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:23 smithi151 bash[22565]: audit 2024-09-10T18:41:23.800684+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:24.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:22.235464+0000 mgr.y (mgr.24512) 292 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: cluster 2024-09-10T18:41:22.801243+0000 mon.a (mon.0) 1990 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.349401+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.754767+0000 mon.b (mon.1) 169 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: cluster 2024-09-10T18:41:23.769746+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.773275+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.796714+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.797546+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.798760+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.799281+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.800156+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[18597]: audit 2024-09-10T18:41:23.800684+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:22.235464+0000 mgr.y (mgr.24512) 292 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: cluster 2024-09-10T18:41:22.801243+0000 mon.a (mon.0) 1990 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.349401+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.100:0/783795957' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-72593-39"}]': finished 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.754767+0000 mon.b (mon.1) 169 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: cluster 2024-09-10T18:41:23.769746+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.773275+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.796714+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.797546+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.798760+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.799281+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:24.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.800156+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:24.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:23 smithi100 bash[31607]: audit 2024-09-10T18:41:23.800684+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:24.826 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:25.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: cluster 2024-09-10T18:41:23.885078+0000 mgr.y (mgr.24512) 293 : cluster [DBG] pgmap v373: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:25.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.333616+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.336224+0000 mon.c (mon.2) 348 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.340119+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.351336+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.351738+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.351943+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: cluster 2024-09-10T18:41:24.352072+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.353591+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.356816+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:25.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:24 smithi151 bash[22565]: audit 2024-09-10T18:41:24.358230+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:25.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: cluster 2024-09-10T18:41:23.885078+0000 mgr.y (mgr.24512) 293 : cluster [DBG] pgmap v373: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.333616+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.336224+0000 mon.c (mon.2) 348 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.340119+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.351336+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.351738+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.351943+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: cluster 2024-09-10T18:41:24.352072+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.353591+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.356816+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[18597]: audit 2024-09-10T18:41:24.358230+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: cluster 2024-09-10T18:41:23.885078+0000 mgr.y (mgr.24512) 293 : cluster [DBG] pgmap v373: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.333616+0000 mon.a (mon.0) 1997 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.336224+0000 mon.c (mon.2) 348 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.340119+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.351336+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-72601-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.351738+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.351943+0000 mon.a (mon.0) 2000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-72593-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: cluster 2024-09-10T18:41:24.352072+0000 mon.a (mon.0) 2001 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.353591+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:25.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.356816+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:25.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:24 smithi100 bash[31607]: audit 2024-09-10T18:41:24.358230+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:26 smithi151 bash[22565]: audit 2024-09-10T18:41:25.775018+0000 mon.b (mon.1) 171 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:26 smithi151 bash[22565]: audit 2024-09-10T18:41:25.790506+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:26 smithi151 bash[22565]: cluster 2024-09-10T18:41:25.790988+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:41:26.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:26 smithi151 bash[22565]: audit 2024-09-10T18:41:25.793992+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:26.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:26 smithi151 bash[22565]: cluster 2024-09-10T18:41:25.885916+0000 mgr.y (mgr.24512) 294 : cluster [DBG] pgmap v376: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:41:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[18597]: audit 2024-09-10T18:41:25.775018+0000 mon.b (mon.1) 171 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[18597]: audit 2024-09-10T18:41:25.790506+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[18597]: cluster 2024-09-10T18:41:25.790988+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:41:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[18597]: audit 2024-09-10T18:41:25.793992+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[18597]: cluster 2024-09-10T18:41:25.885916+0000 mgr.y (mgr.24512) 294 : cluster [DBG] pgmap v376: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:41:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[31607]: audit 2024-09-10T18:41:25.775018+0000 mon.b (mon.1) 171 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[31607]: audit 2024-09-10T18:41:25.790506+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[31607]: cluster 2024-09-10T18:41:25.790988+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:41:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[31607]: audit 2024-09-10T18:41:25.793992+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:26 smithi100 bash[31607]: cluster 2024-09-10T18:41:25.885916+0000 mgr.y (mgr.24512) 294 : cluster [DBG] pgmap v376: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:41:27.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:27 smithi151 bash[22565]: audit 2024-09-10T18:41:26.351996+0000 mon.b (mon.1) 172 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]: dispatch 2024-09-10T18:41:27.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:27 smithi151 bash[22565]: audit 2024-09-10T18:41:26.353343+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:27.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:27 smithi151 bash[22565]: audit 2024-09-10T18:41:26.365488+0000 mon.a (mon.0) 2007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:27.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:27 smithi151 bash[22565]: audit 2024-09-10T18:41:26.365621+0000 mon.a (mon.0) 2008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]': finished 2024-09-10T18:41:27.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:27 smithi151 bash[22565]: cluster 2024-09-10T18:41:26.365686+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:41:27.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:27 smithi151 bash[22565]: audit 2024-09-10T18:41:26.372067+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]: dispatch 2024-09-10T18:41:27.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:27 smithi151 bash[22565]: audit 2024-09-10T18:41:26.372296+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:27.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[18597]: audit 2024-09-10T18:41:26.351996+0000 mon.b (mon.1) 172 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]: dispatch 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[18597]: audit 2024-09-10T18:41:26.353343+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[18597]: audit 2024-09-10T18:41:26.365488+0000 mon.a (mon.0) 2007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[18597]: audit 2024-09-10T18:41:26.365621+0000 mon.a (mon.0) 2008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]': finished 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[18597]: cluster 2024-09-10T18:41:26.365686+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[18597]: audit 2024-09-10T18:41:26.372067+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]: dispatch 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[18597]: audit 2024-09-10T18:41:26.372296+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[31607]: audit 2024-09-10T18:41:26.351996+0000 mon.b (mon.1) 172 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]: dispatch 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[31607]: audit 2024-09-10T18:41:26.353343+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[31607]: audit 2024-09-10T18:41:26.365488+0000 mon.a (mon.0) 2007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-72593-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[31607]: audit 2024-09-10T18:41:26.365621+0000 mon.a (mon.0) 2008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-33"}]': finished 2024-09-10T18:41:27.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[31607]: cluster 2024-09-10T18:41:26.365686+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:41:27.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[31607]: audit 2024-09-10T18:41:26.372067+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]: dispatch 2024-09-10T18:41:27.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:27 smithi100 bash[31607]: audit 2024-09-10T18:41:26.372296+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:28.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: cluster 2024-09-10T18:41:27.361341+0000 mon.a (mon.0) 2012 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: audit 2024-09-10T18:41:27.362116+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: audit 2024-09-10T18:41:27.375594+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]': finished 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: audit 2024-09-10T18:41:27.375704+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: cluster 2024-09-10T18:41:27.375801+0000 mon.a (mon.0) 2015 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: audit 2024-09-10T18:41:27.379872+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: audit 2024-09-10T18:41:27.546356+0000 mon.b (mon.1) 175 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: audit 2024-09-10T18:41:27.564445+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:28.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:28 smithi151 bash[22565]: cluster 2024-09-10T18:41:27.886741+0000 mgr.y (mgr.24512) 295 : cluster [DBG] pgmap v379: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: cluster 2024-09-10T18:41:27.361341+0000 mon.a (mon.0) 2012 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: audit 2024-09-10T18:41:27.362116+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: audit 2024-09-10T18:41:27.375594+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]': finished 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: audit 2024-09-10T18:41:27.375704+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: cluster 2024-09-10T18:41:27.375801+0000 mon.a (mon.0) 2015 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: audit 2024-09-10T18:41:27.379872+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: audit 2024-09-10T18:41:27.546356+0000 mon.b (mon.1) 175 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: audit 2024-09-10T18:41:27.564445+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[18597]: cluster 2024-09-10T18:41:27.886741+0000 mgr.y (mgr.24512) 295 : cluster [DBG] pgmap v379: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: cluster 2024-09-10T18:41:27.361341+0000 mon.a (mon.0) 2012 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: audit 2024-09-10T18:41:27.362116+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.100:0/3063434608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: audit 2024-09-10T18:41:27.375594+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-33", "mode": "writeback"}]': finished 2024-09-10T18:41:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: audit 2024-09-10T18:41:27.375704+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: cluster 2024-09-10T18:41:27.375801+0000 mon.a (mon.0) 2015 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:41:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: audit 2024-09-10T18:41:27.379872+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]: dispatch 2024-09-10T18:41:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: audit 2024-09-10T18:41:27.546356+0000 mon.b (mon.1) 175 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: audit 2024-09-10T18:41:27.564445+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:28.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:28 smithi100 bash[31607]: cluster 2024-09-10T18:41:27.886741+0000 mgr.y (mgr.24512) 295 : cluster [DBG] pgmap v379: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:29.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.364438+0000 mon.b (mon.1) 176 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:29.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.379835+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:29.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.379969+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:29.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: cluster 2024-09-10T18:41:28.380043+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:41:29.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.382466+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:29.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.387775+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.388159+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.408752+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.409182+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.410483+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.410844+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.411452+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:29 smithi151 bash[22565]: audit 2024-09-10T18:41:28.411826+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.364438+0000 mon.b (mon.1) 176 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.379835+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.379969+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: cluster 2024-09-10T18:41:28.380043+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.382466+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.387775+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.388159+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.408752+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.409182+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.410483+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.410844+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.411452+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[18597]: audit 2024-09-10T18:41:28.411826+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.364438+0000 mon.b (mon.1) 176 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.379835+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-72601-49"}]': finished 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.379969+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: cluster 2024-09-10T18:41:28.380043+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.382466+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.387775+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.388159+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.408752+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.409182+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.410483+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.410844+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.411452+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:29 smithi100 bash[31607]: audit 2024-09-10T18:41:28.411826+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:30.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: cluster 2024-09-10T18:41:29.376489+0000 mon.a (mon.0) 2026 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:30.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: audit 2024-09-10T18:41:29.385617+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]': finished 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: audit 2024-09-10T18:41:29.385804+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: audit 2024-09-10T18:41:29.385987+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: cluster 2024-09-10T18:41:29.386178+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: audit 2024-09-10T18:41:29.388579+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: audit 2024-09-10T18:41:29.388806+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: audit 2024-09-10T18:41:29.395884+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: audit 2024-09-10T18:41:29.396315+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: cluster 2024-09-10T18:41:29.887637+0000 mgr.y (mgr.24512) 296 : cluster [DBG] pgmap v382: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:30.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:30 smithi151 bash[22565]: cluster 2024-09-10T18:41:30.331966+0000 mon.a (mon.0) 2033 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: cluster 2024-09-10T18:41:29.376489+0000 mon.a (mon.0) 2026 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: audit 2024-09-10T18:41:29.385617+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]': finished 2024-09-10T18:41:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: audit 2024-09-10T18:41:29.385804+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:30.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: audit 2024-09-10T18:41:29.385987+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: cluster 2024-09-10T18:41:29.386178+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: audit 2024-09-10T18:41:29.388579+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: audit 2024-09-10T18:41:29.388806+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: audit 2024-09-10T18:41:29.395884+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: audit 2024-09-10T18:41:29.396315+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: cluster 2024-09-10T18:41:29.887637+0000 mgr.y (mgr.24512) 296 : cluster [DBG] pgmap v382: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[18597]: cluster 2024-09-10T18:41:30.331966+0000 mon.a (mon.0) 2033 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: cluster 2024-09-10T18:41:29.376489+0000 mon.a (mon.0) 2026 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: audit 2024-09-10T18:41:29.385617+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-33"}]': finished 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: audit 2024-09-10T18:41:29.385804+0000 mon.a (mon.0) 2028 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: audit 2024-09-10T18:41:29.385987+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-72601-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: cluster 2024-09-10T18:41:29.386178+0000 mon.a (mon.0) 2030 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:41:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: audit 2024-09-10T18:41:29.388579+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.100:0/1989725819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: audit 2024-09-10T18:41:29.388806+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: audit 2024-09-10T18:41:29.395884+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]: dispatch 2024-09-10T18:41:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: audit 2024-09-10T18:41:29.396315+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: cluster 2024-09-10T18:41:29.887637+0000 mgr.y (mgr.24512) 296 : cluster [DBG] pgmap v382: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:30.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:30 smithi100 bash[31607]: cluster 2024-09-10T18:41:30.331966+0000 mon.a (mon.0) 2033 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:31.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:31 smithi151 bash[22565]: audit 2024-09-10T18:41:30.407685+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:31.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:31 smithi151 bash[22565]: cluster 2024-09-10T18:41:30.836973+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:41:31.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:31 smithi151 bash[22565]: audit 2024-09-10T18:41:31.258918+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:31.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:31 smithi151 bash[22565]: audit 2024-09-10T18:41:31.259648+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:31.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:31 smithi151 bash[22565]: audit 2024-09-10T18:41:31.260304+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-72593-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:31] "GET /metrics HTTP/1.1" 200 229966 "" "Prometheus/2.43.0" 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[18597]: audit 2024-09-10T18:41:30.407685+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[18597]: cluster 2024-09-10T18:41:30.836973+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[18597]: audit 2024-09-10T18:41:31.258918+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[18597]: audit 2024-09-10T18:41:31.259648+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[18597]: audit 2024-09-10T18:41:31.260304+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-72593-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[31607]: audit 2024-09-10T18:41:30.407685+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-72593-40"}]': finished 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[31607]: cluster 2024-09-10T18:41:30.836973+0000 mon.a (mon.0) 2035 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[31607]: audit 2024-09-10T18:41:31.258918+0000 mon.a (mon.0) 2036 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[31607]: audit 2024-09-10T18:41:31.259648+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:31 smithi100 bash[31607]: audit 2024-09-10T18:41:31.260304+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-72593-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:32.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:41:33.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[22565]: audit 2024-09-10T18:41:31.413415+0000 mon.b (mon.1) 177 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[22565]: audit 2024-09-10T18:41:31.427757+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:33.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[22565]: audit 2024-09-10T18:41:31.427941+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-72593-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:33.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[22565]: cluster 2024-09-10T18:41:31.428011+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:41:33.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[22565]: audit 2024-09-10T18:41:31.429788+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-72593-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:33.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[22565]: audit 2024-09-10T18:41:31.431465+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:32 smithi151 bash[22565]: cluster 2024-09-10T18:41:31.888517+0000 mgr.y (mgr.24512) 297 : cluster [DBG] pgmap v385: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:33.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[31607]: audit 2024-09-10T18:41:31.413415+0000 mon.b (mon.1) 177 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[31607]: audit 2024-09-10T18:41:31.427757+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[31607]: audit 2024-09-10T18:41:31.427941+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-72593-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[31607]: cluster 2024-09-10T18:41:31.428011+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[31607]: audit 2024-09-10T18:41:31.429788+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-72593-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[31607]: audit 2024-09-10T18:41:31.431465+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[31607]: cluster 2024-09-10T18:41:31.888517+0000 mgr.y (mgr.24512) 297 : cluster [DBG] pgmap v385: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[18597]: audit 2024-09-10T18:41:31.413415+0000 mon.b (mon.1) 177 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[18597]: audit 2024-09-10T18:41:31.427757+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-72601-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[18597]: audit 2024-09-10T18:41:31.427941+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-72593-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:33.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[18597]: cluster 2024-09-10T18:41:31.428011+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:41:33.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[18597]: audit 2024-09-10T18:41:31.429788+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-72593-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:33.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[18597]: audit 2024-09-10T18:41:31.431465+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:32 smithi100 bash[18597]: cluster 2024-09-10T18:41:31.888517+0000 mgr.y (mgr.24512) 297 : cluster [DBG] pgmap v385: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:34.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:32.247146+0000 mgr.y (mgr.24512) 298 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:34.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:32.828196+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:34.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:32.842763+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:34.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: cluster 2024-09-10T18:41:32.843267+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:32.856377+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:33.417532+0000 mon.b (mon.1) 179 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:33.432052+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-72593-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:33.432178+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: cluster 2024-09-10T18:41:33.432250+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:33.434972+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:33.436031+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:33 smithi151 bash[22565]: audit 2024-09-10T18:41:33.437440+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:34.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:32.247146+0000 mgr.y (mgr.24512) 298 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:32.828196+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:32.842763+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: cluster 2024-09-10T18:41:32.843267+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:32.856377+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:33.417532+0000 mon.b (mon.1) 179 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:33.432052+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-72593-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:33.432178+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: cluster 2024-09-10T18:41:33.432250+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:33.434972+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:33.436031+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[31607]: audit 2024-09-10T18:41:33.437440+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:32.247146+0000 mgr.y (mgr.24512) 298 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:32.828196+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:32.842763+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: cluster 2024-09-10T18:41:32.843267+0000 mon.a (mon.0) 2045 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:32.856377+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:33.417532+0000 mon.b (mon.1) 179 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:33.432052+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-72593-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:33.432178+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: cluster 2024-09-10T18:41:33.432250+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:33.434972+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:33.436031+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:34.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:33 smithi100 bash[18597]: audit 2024-09-10T18:41:33.437440+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:34.858 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:35.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: cluster 2024-09-10T18:41:33.889407+0000 mgr.y (mgr.24512) 299 : cluster [DBG] pgmap v388: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:35.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: audit 2024-09-10T18:41:34.421654+0000 mon.b (mon.1) 180 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]: dispatch 2024-09-10T18:41:35.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: audit 2024-09-10T18:41:34.434340+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:35.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: audit 2024-09-10T18:41:34.434621+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:35.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: cluster 2024-09-10T18:41:34.434879+0000 mon.a (mon.0) 2054 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:41:35.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: audit 2024-09-10T18:41:34.441537+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]: dispatch 2024-09-10T18:41:35.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: audit 2024-09-10T18:41:34.442216+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:35.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:34 smithi151 bash[22565]: audit 2024-09-10T18:41:34.442639+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: cluster 2024-09-10T18:41:33.889407+0000 mgr.y (mgr.24512) 299 : cluster [DBG] pgmap v388: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:35.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: audit 2024-09-10T18:41:34.421654+0000 mon.b (mon.1) 180 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]: dispatch 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: audit 2024-09-10T18:41:34.434340+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: audit 2024-09-10T18:41:34.434621+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: cluster 2024-09-10T18:41:34.434879+0000 mon.a (mon.0) 2054 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: audit 2024-09-10T18:41:34.441537+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]: dispatch 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: audit 2024-09-10T18:41:34.442216+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[18597]: audit 2024-09-10T18:41:34.442639+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:35.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: cluster 2024-09-10T18:41:33.889407+0000 mgr.y (mgr.24512) 299 : cluster [DBG] pgmap v388: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: audit 2024-09-10T18:41:34.421654+0000 mon.b (mon.1) 180 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]: dispatch 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: audit 2024-09-10T18:41:34.434340+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: audit 2024-09-10T18:41:34.434621+0000 mon.a (mon.0) 2053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: cluster 2024-09-10T18:41:34.434879+0000 mon.a (mon.0) 2054 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: audit 2024-09-10T18:41:34.441537+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]: dispatch 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: audit 2024-09-10T18:41:34.442216+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.100:0/723740160' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:34 smithi100 bash[31607]: audit 2024-09-10T18:41:34.442639+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]: dispatch 2024-09-10T18:41:36.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:35 smithi151 bash[22565]: cluster 2024-09-10T18:41:35.428943+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:36.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:35 smithi151 bash[22565]: audit 2024-09-10T18:41:35.438098+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]': finished 2024-09-10T18:41:36.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:35 smithi151 bash[22565]: audit 2024-09-10T18:41:35.438365+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:36.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:35 smithi151 bash[22565]: cluster 2024-09-10T18:41:35.438577+0000 mon.a (mon.0) 2060 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:41:36.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:35 smithi151 bash[22565]: audit 2024-09-10T18:41:35.440347+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[18597]: cluster 2024-09-10T18:41:35.428943+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[18597]: audit 2024-09-10T18:41:35.438098+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]': finished 2024-09-10T18:41:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[18597]: audit 2024-09-10T18:41:35.438365+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:36.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[18597]: cluster 2024-09-10T18:41:35.438577+0000 mon.a (mon.0) 2060 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:41:36.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[18597]: audit 2024-09-10T18:41:35.440347+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[31607]: cluster 2024-09-10T18:41:35.428943+0000 mon.a (mon.0) 2057 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[31607]: audit 2024-09-10T18:41:35.438098+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-35", "mode": "writeback"}]': finished 2024-09-10T18:41:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[31607]: audit 2024-09-10T18:41:35.438365+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-72601-50"}]': finished 2024-09-10T18:41:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[31607]: cluster 2024-09-10T18:41:35.438577+0000 mon.a (mon.0) 2060 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:41:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:35 smithi100 bash[31607]: audit 2024-09-10T18:41:35.440347+0000 mon.a (mon.0) 2061 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:37.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: cluster 2024-09-10T18:41:35.890151+0000 mgr.y (mgr.24512) 300 : cluster [DBG] pgmap v391: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:37.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.403061+0000 mon.c (mon.2) 358 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:37.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: cluster 2024-09-10T18:41:36.431369+0000 mon.a (mon.0) 2062 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:37.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.448271+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:37.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: cluster 2024-09-10T18:41:36.448473+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:41:37.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.452224+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:37.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.455291+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.100:0/1033225923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.457234+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.856570+0000 mon.c (mon.2) 360 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:37.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.858048+0000 mon.c (mon.2) 361 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:37.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:36 smithi151 bash[22565]: audit 2024-09-10T18:41:36.867317+0000 mon.a (mon.0) 2067 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: cluster 2024-09-10T18:41:35.890151+0000 mgr.y (mgr.24512) 300 : cluster [DBG] pgmap v391: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.403061+0000 mon.c (mon.2) 358 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: cluster 2024-09-10T18:41:36.431369+0000 mon.a (mon.0) 2062 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.448271+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: cluster 2024-09-10T18:41:36.448473+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.452224+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.455291+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.100:0/1033225923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.457234+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.856570+0000 mon.c (mon.2) 360 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.858048+0000 mon.c (mon.2) 361 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[18597]: audit 2024-09-10T18:41:36.867317+0000 mon.a (mon.0) 2067 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: cluster 2024-09-10T18:41:35.890151+0000 mgr.y (mgr.24512) 300 : cluster [DBG] pgmap v391: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.403061+0000 mon.c (mon.2) 358 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: cluster 2024-09-10T18:41:36.431369+0000 mon.a (mon.0) 2062 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.448271+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: cluster 2024-09-10T18:41:36.448473+0000 mon.a (mon.0) 2064 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.452224+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]: dispatch 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.455291+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.100:0/1033225923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.457234+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.856570+0000 mon.c (mon.2) 360 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.858048+0000 mon.c (mon.2) 361 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:37.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:36 smithi100 bash[31607]: audit 2024-09-10T18:41:36.867317+0000 mon.a (mon.0) 2067 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:37.461 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: waiting for 1297,1298,1299,1300,1301,1302,1303,1304,1305,1306,1307,1308,1309,1310,1311,1312,1313,1314,1315,1316,1317,1318,1319,1320,1321,1322,1323,1324,1325,1326,1327,1328,1329,1330,1331,1332,1333,1334,1335,1336,1337,1338,1339,1340,1341,1342,1343,1344,1345,1346,1347,1348,1349,1350,1351,1352,1353,1354,1355,1356,1357,1358,1359,1360,1361,1362,1363,1364,1365,1366,1367,1368,1369,1370,1371,1372,1373,1374,1375,1376,1377,1378,1379,1380,1381,1382,1383,1384,1385,1386,1387,1388,1389,1390,1391,1392,1393,1394,1395,1396,1397,1398,1399,1400,1401,1402,1403,1404,1405,1406,1407,1408,1409,1410,1411,1412,1413,1414,1415,1416,1417,1418,1419,1420,1421,1422,1423,1424,1425,1426,1427,1428,1429,1430,1431,1432,1433,1434,1435,1436,1437,1438,1439,1440,1441,1442,1443,1444,1445,1446,1447,1448,1449,1450,1451,1452,1453,1454,1455,1456,1457,1458,1459,1460,1461,1462,1463,1464,1465,1466,1467,1468,1469,1470,1471,1472,1473,1474,1475,1476,1477,1478,1479,1480,1481,1482,1483,1484,1485,1486,1487,1488,1489,1490,1491,1492,1493,1494,1495,1496,1497,1498,1499,1500,1501,1502,1503,1504,1505,1506,1507,1508,1509,1510,1511,1512,1513,1514,1515,1516,1517,1518,1519,1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539,1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559,1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577,1578,1579,1580,1581,1582,1583,1584,1585,1586,1587,1588,1589,1590,1591,1592,1593,1594,1595,1596,1597,1598,1599,1600,1601,1602,1603,1604,1605,1606,1607,1608,1609,1610,1611,1612,1613,1614,1615,1616,1617,1618,1619,1620,1621,1622,1623,1624,1625,1626,1627,1628,1629,1630,1631,1632,1633,1634,1635,1636,1637,1638,1639,1640,1641,1642,1643,1644,1645,1646,1647,1648,1649,1650,1651,1652,1653,1654,1655,1656,1657,1658,1659,1660,1661,1662,1663,1664,1665,1666,1667,1668,1669,1670,1671,1672,1673,1674,1675,1676,1677,1678,1679,1680,1681,1682,1683,1684,1685,1686,1687,1688,1689,1690,1691,1692,1693,1694,1695,1696,1697,1698,1699,1700,1701,1702,1703,1704,1705,1706,1707,1708,1709,1710,1711,1712,1713,1714,1715,1716,1717,1718,1719,1720,1721,1722,1723,1724,1725,1726,1727,1728,1729,1730,1731,1732,1733,1734,1735,1736,1737,1738,1739,1740,1741,1742,1743,1744,1745,1746,1747,1748,1749,1750,1751,1752,1753,1754,1755,1756,1757,1758,1759,1760,1761,1762,1763,1764,1765,1766,1767,1768,1769,1770,1771,1772,1773,1774,1775,1776,1777,1778,1779,1780,1781,1782,1783,1784,1785,1786,1787,1788,1789,1790,1791,1792,1793,1794,1795,1796,1797,1798,1799,1800,1801,1802,1803,1804,1805,1806,1807,1808,1809,1810,1811,1812,1813,1814,1815,1816,1817,1818,1819,1820,1821,1822,1823,1824,1825,1826,1827,1828,1829,1830,1831,1832,1833,1834,1835,1836,1837,1838,1839,1840,1841,1842,1843,1844,1845,1846,1847,1848,1849,1850,1851,1852,1853,1854,1855,1856,1857,1858,1859,1860,1861,1862,1863,1864,1865,1866,1867,1868,1869,1870,1871,1872,1873,1874,1875,1876,1877,1878,1879,1880,1881,1882,1883,1884,1885,1886,1887,1888,1889,1890,1891,1892,1893,1894,1895,1896,1897,1898,1899,1900,1901,1902,1903,1904,1905,1906,1907,1908,1909,1910,1911,1912,1913,1914,1915,1916,1917,1918,1919,1920,1921,1922,1923,1924,1925,1926,1927,1928,1929,1930,1931,1932,1933,1934,1935,1936,1937,1938,1939,1940,1941,1942,1943,1944,1945,1946,1947,1948,1949,1950,1951,1952,1953,1954,1955,1956,1957,1958,1959,1960,1961,1962,1963,1964,1965,1966,1967,1968,1969,1970,1971,1972,1973,1974,1975,1976,1977,1978,1979,1980,1981,1982,1983,1984,1985,1986,1987,1988,1989,1990,199ete 2024-09-10T18:41:37.461 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7031 ms) 2024-09-10T18:41:37.461 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-10T18:41:37.461 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6761 ms) 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7001 ms) 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7055 ms) 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7029 ms) 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7051 ms) 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7032 ms) 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6855 ms) 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-10T18:41:37.462 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7012 ms) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7450 ms) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6545 ms) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7410 ms) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7001 ms) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (6637 ms) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7048 ms) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111962 ms total) 2024-09-10T18:41:37.463 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] Global test environment tear-down 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (203170 ms total) 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ PASSED ] 41 tests. 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73008 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 73008 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.464 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72650 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72650 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72999 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72999 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72964 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72964 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72604 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72604 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72762 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72762 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72910 2024-09-10T18:41:37.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72910 2024-09-10T18:41:37.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:41:37.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72749 2024-09-10T18:41:37.466 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72749 2024-09-10T18:41:38.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[18597]: audit 2024-09-10T18:41:37.452993+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:38.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[18597]: audit 2024-09-10T18:41:37.453157+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[18597]: cluster 2024-09-10T18:41:37.453760+0000 mon.a (mon.0) 2070 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[18597]: audit 2024-09-10T18:41:37.501787+0000 mon.b (mon.1) 181 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[18597]: audit 2024-09-10T18:41:37.519667+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[18597]: cluster 2024-09-10T18:41:37.891027+0000 mgr.y (mgr.24512) 301 : cluster [DBG] pgmap v394: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[31607]: audit 2024-09-10T18:41:37.452993+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[31607]: audit 2024-09-10T18:41:37.453157+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[31607]: cluster 2024-09-10T18:41:37.453760+0000 mon.a (mon.0) 2070 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[31607]: audit 2024-09-10T18:41:37.501787+0000 mon.b (mon.1) 181 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[31607]: audit 2024-09-10T18:41:37.519667+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:38 smithi100 bash[31607]: cluster 2024-09-10T18:41:37.891027+0000 mgr.y (mgr.24512) 301 : cluster [DBG] pgmap v394: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:38.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:38 smithi151 bash[22565]: audit 2024-09-10T18:41:37.452993+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? 172.21.15.100:0/332089682' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-72593-41"}]': finished 2024-09-10T18:41:38.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:38 smithi151 bash[22565]: audit 2024-09-10T18:41:37.453157+0000 mon.a (mon.0) 2069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-72601-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:38.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:38 smithi151 bash[22565]: cluster 2024-09-10T18:41:37.453760+0000 mon.a (mon.0) 2070 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:41:38.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:38 smithi151 bash[22565]: audit 2024-09-10T18:41:37.501787+0000 mon.b (mon.1) 181 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:38.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:38 smithi151 bash[22565]: audit 2024-09-10T18:41:37.519667+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:38.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:38 smithi151 bash[22565]: cluster 2024-09-10T18:41:37.891027+0000 mgr.y (mgr.24512) 301 : cluster [DBG] pgmap v394: 260 pgs: 32 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.461140+0000 mon.b (mon.1) 182 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.461316+0000 mon.a (mon.0) 2072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: cluster 2024-09-10T18:41:38.461474+0000 mon.a (mon.0) 2073 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.478978+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.487115+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.487472+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.488363+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.488675+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.489207+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:38.489528+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:39.334345+0000 mon.a (mon.0) 2078 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[18597]: audit 2024-09-10T18:41:39.336706+0000 mon.c (mon.2) 365 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.461140+0000 mon.b (mon.1) 182 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.461316+0000 mon.a (mon.0) 2072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: cluster 2024-09-10T18:41:38.461474+0000 mon.a (mon.0) 2073 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.478978+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.487115+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.487472+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.488363+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.488675+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.489207+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:38.489528+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:39.334345+0000 mon.a (mon.0) 2078 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:39 smithi100 bash[31607]: audit 2024-09-10T18:41:39.336706+0000 mon.c (mon.2) 365 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:39.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.461140+0000 mon.b (mon.1) 182 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.461316+0000 mon.a (mon.0) 2072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: cluster 2024-09-10T18:41:38.461474+0000 mon.a (mon.0) 2073 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.478978+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.487115+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.487472+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.488363+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.488675+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.489207+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:38.489528+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:39.334345+0000 mon.a (mon.0) 2078 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:39.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:39 smithi151 bash[22565]: audit 2024-09-10T18:41:39.336706+0000 mon.c (mon.2) 365 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:40.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[18597]: audit 2024-09-10T18:41:39.467679+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:40.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[18597]: audit 2024-09-10T18:41:39.467918+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:40.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[18597]: cluster 2024-09-10T18:41:39.468139+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:41:40.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[18597]: audit 2024-09-10T18:41:39.470571+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[18597]: audit 2024-09-10T18:41:39.478170+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[18597]: cluster 2024-09-10T18:41:39.891846+0000 mgr.y (mgr.24512) 302 : cluster [DBG] pgmap v397: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[31607]: audit 2024-09-10T18:41:39.467679+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[31607]: audit 2024-09-10T18:41:39.467918+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[31607]: cluster 2024-09-10T18:41:39.468139+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[31607]: audit 2024-09-10T18:41:39.470571+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[31607]: audit 2024-09-10T18:41:39.478170+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:40 smithi100 bash[31607]: cluster 2024-09-10T18:41:39.891846+0000 mgr.y (mgr.24512) 302 : cluster [DBG] pgmap v397: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:40.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:40 smithi151 bash[22565]: audit 2024-09-10T18:41:39.467679+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:40.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:40 smithi151 bash[22565]: audit 2024-09-10T18:41:39.467918+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:40.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:40 smithi151 bash[22565]: cluster 2024-09-10T18:41:39.468139+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:41:40.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:40 smithi151 bash[22565]: audit 2024-09-10T18:41:39.470571+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:40.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:40 smithi151 bash[22565]: audit 2024-09-10T18:41:39.478170+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:40.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:40 smithi151 bash[22565]: cluster 2024-09-10T18:41:39.891846+0000 mgr.y (mgr.24512) 302 : cluster [DBG] pgmap v397: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:41:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:41] "GET /metrics HTTP/1.1" 200 228463 "" "Prometheus/2.43.0" 2024-09-10T18:41:41.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:41 smithi100 bash[18597]: cluster 2024-09-10T18:41:40.928990+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:41:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:41 smithi100 bash[18597]: audit 2024-09-10T18:41:40.937776+0000 mon.b (mon.1) 183 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:41 smithi100 bash[18597]: audit 2024-09-10T18:41:40.956021+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:41 smithi100 bash[31607]: cluster 2024-09-10T18:41:40.928990+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:41:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:41 smithi100 bash[31607]: audit 2024-09-10T18:41:40.937776+0000 mon.b (mon.1) 183 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:41 smithi100 bash[31607]: audit 2024-09-10T18:41:40.956021+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:41.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:41 smithi151 bash[22565]: cluster 2024-09-10T18:41:40.928990+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:41:41.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:41 smithi151 bash[22565]: audit 2024-09-10T18:41:40.937776+0000 mon.b (mon.1) 183 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:41.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:41 smithi151 bash[22565]: audit 2024-09-10T18:41:40.956021+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:42.516 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:41:42.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: audit 2024-09-10T18:41:41.490780+0000 mon.b (mon.1) 184 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:42.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: audit 2024-09-10T18:41:41.505654+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:42.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: audit 2024-09-10T18:41:41.505874+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:42.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: cluster 2024-09-10T18:41:41.506079+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:41:42.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: audit 2024-09-10T18:41:41.509890+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: cluster 2024-09-10T18:41:41.892696+0000 mgr.y (mgr.24512) 303 : cluster [DBG] pgmap v400: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: cluster 2024-09-10T18:41:42.493551+0000 mon.a (mon.0) 2089 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: audit 2024-09-10T18:41:42.503017+0000 mon.a (mon.0) 2090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[31607]: cluster 2024-09-10T18:41:42.503217+0000 mon.a (mon.0) 2091 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: audit 2024-09-10T18:41:41.490780+0000 mon.b (mon.1) 184 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: audit 2024-09-10T18:41:41.505654+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: audit 2024-09-10T18:41:41.505874+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: cluster 2024-09-10T18:41:41.506079+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: audit 2024-09-10T18:41:41.509890+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: cluster 2024-09-10T18:41:41.892696+0000 mgr.y (mgr.24512) 303 : cluster [DBG] pgmap v400: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:41:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: cluster 2024-09-10T18:41:42.493551+0000 mon.a (mon.0) 2089 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:42.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: audit 2024-09-10T18:41:42.503017+0000 mon.a (mon.0) 2090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:42.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:42 smithi100 bash[18597]: cluster 2024-09-10T18:41:42.503217+0000 mon.a (mon.0) 2091 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:41:42.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: audit 2024-09-10T18:41:41.490780+0000 mon.b (mon.1) 184 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: audit 2024-09-10T18:41:41.505654+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-72601-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: audit 2024-09-10T18:41:41.505874+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: cluster 2024-09-10T18:41:41.506079+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: audit 2024-09-10T18:41:41.509890+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]: dispatch 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: cluster 2024-09-10T18:41:41.892696+0000 mgr.y (mgr.24512) 303 : cluster [DBG] pgmap v400: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: cluster 2024-09-10T18:41:42.493551+0000 mon.a (mon.0) 2089 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: audit 2024-09-10T18:41:42.503017+0000 mon.a (mon.0) 2090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-35"}]': finished 2024-09-10T18:41:42.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:42 smithi151 bash[22565]: cluster 2024-09-10T18:41:42.503217+0000 mon.a (mon.0) 2091 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:41:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[31607]: audit 2024-09-10T18:41:42.257737+0000 mgr.y (mgr.24512) 304 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[31607]: cluster 2024-09-10T18:41:42.514073+0000 mon.a (mon.0) 2092 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[31607]: cluster 2024-09-10T18:41:43.505606+0000 mon.a (mon.0) 2093 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:41:44.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[31607]: audit 2024-09-10T18:41:43.508533+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[31607]: audit 2024-09-10T18:41:43.512047+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[18597]: audit 2024-09-10T18:41:42.257737+0000 mgr.y (mgr.24512) 304 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[18597]: cluster 2024-09-10T18:41:42.514073+0000 mon.a (mon.0) 2092 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[18597]: cluster 2024-09-10T18:41:43.505606+0000 mon.a (mon.0) 2093 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:41:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[18597]: audit 2024-09-10T18:41:43.508533+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:43 smithi100 bash[18597]: audit 2024-09-10T18:41:43.512047+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:44.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:43 smithi151 bash[22565]: audit 2024-09-10T18:41:42.257737+0000 mgr.y (mgr.24512) 304 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:44.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:43 smithi151 bash[22565]: cluster 2024-09-10T18:41:42.514073+0000 mon.a (mon.0) 2092 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:44.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:43 smithi151 bash[22565]: cluster 2024-09-10T18:41:43.505606+0000 mon.a (mon.0) 2093 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:41:44.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:43 smithi151 bash[22565]: audit 2024-09-10T18:41:43.508533+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:44.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:43 smithi151 bash[22565]: audit 2024-09-10T18:41:43.512047+0000 mon.a (mon.0) 2094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:44.929 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[18597]: cluster 2024-09-10T18:41:43.893458+0000 mgr.y (mgr.24512) 305 : cluster [DBG] pgmap v403: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[18597]: audit 2024-09-10T18:41:44.503006+0000 mon.b (mon.1) 185 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[18597]: audit 2024-09-10T18:41:44.514759+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[18597]: cluster 2024-09-10T18:41:44.515084+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[18597]: audit 2024-09-10T18:41:44.517895+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[18597]: audit 2024-09-10T18:41:44.518710+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[18597]: audit 2024-09-10T18:41:44.520992+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[31607]: cluster 2024-09-10T18:41:43.893458+0000 mgr.y (mgr.24512) 305 : cluster [DBG] pgmap v403: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[31607]: audit 2024-09-10T18:41:44.503006+0000 mon.b (mon.1) 185 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[31607]: audit 2024-09-10T18:41:44.514759+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[31607]: cluster 2024-09-10T18:41:44.515084+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:41:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[31607]: audit 2024-09-10T18:41:44.517895+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[31607]: audit 2024-09-10T18:41:44.518710+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:45.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:44 smithi100 bash[31607]: audit 2024-09-10T18:41:44.520992+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[22565]: cluster 2024-09-10T18:41:43.893458+0000 mgr.y (mgr.24512) 305 : cluster [DBG] pgmap v403: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:41:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[22565]: audit 2024-09-10T18:41:44.503006+0000 mon.b (mon.1) 185 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[22565]: audit 2024-09-10T18:41:44.514759+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[22565]: cluster 2024-09-10T18:41:44.515084+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:41:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[22565]: audit 2024-09-10T18:41:44.517895+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.100:0/2051387533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[22565]: audit 2024-09-10T18:41:44.518710+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]: dispatch 2024-09-10T18:41:45.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:44 smithi151 bash[22565]: audit 2024-09-10T18:41:44.520992+0000 mon.a (mon.0) 2098 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:46.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.508039+0000 mon.b (mon.1) 186 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.518671+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.518876+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: cluster 2024-09-10T18:41:45.519084+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.526911+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.551749+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.552095+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.552976+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.553289+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.553834+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: audit 2024-09-10T18:41:45.554105+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[18597]: cluster 2024-09-10T18:41:45.894251+0000 mgr.y (mgr.24512) 306 : cluster [DBG] pgmap v406: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.508039+0000 mon.b (mon.1) 186 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.518671+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.518876+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: cluster 2024-09-10T18:41:45.519084+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.526911+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.551749+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.552095+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.552976+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.553289+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.553834+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: audit 2024-09-10T18:41:45.554105+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:46.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:46 smithi100 bash[31607]: cluster 2024-09-10T18:41:45.894251+0000 mgr.y (mgr.24512) 306 : cluster [DBG] pgmap v406: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:41:46.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.508039+0000 mon.b (mon.1) 186 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:46.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.518671+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-72601-52"}]': finished 2024-09-10T18:41:46.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.518876+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:46.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: cluster 2024-09-10T18:41:45.519084+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:41:46.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.526911+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:46.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.551749+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.552095+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.552976+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.553289+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.553834+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: audit 2024-09-10T18:41:45.554105+0000 mon.a (mon.0) 2105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:46 smithi151 bash[22565]: cluster 2024-09-10T18:41:45.894251+0000 mgr.y (mgr.24512) 306 : cluster [DBG] pgmap v406: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:41:48.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[18597]: audit 2024-09-10T18:41:46.505894+0000 mon.b (mon.1) 187 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:48.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[18597]: audit 2024-09-10T18:41:46.521897+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[18597]: audit 2024-09-10T18:41:46.522187+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[18597]: cluster 2024-09-10T18:41:46.522457+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[18597]: audit 2024-09-10T18:41:46.523380+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[18597]: audit 2024-09-10T18:41:46.524587+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[18597]: audit 2024-09-10T18:41:46.525308+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[31607]: audit 2024-09-10T18:41:46.505894+0000 mon.b (mon.1) 187 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[31607]: audit 2024-09-10T18:41:46.521897+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[31607]: audit 2024-09-10T18:41:46.522187+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[31607]: cluster 2024-09-10T18:41:46.522457+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:41:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[31607]: audit 2024-09-10T18:41:46.523380+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:48.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[31607]: audit 2024-09-10T18:41:46.524587+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:48.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:47 smithi100 bash[31607]: audit 2024-09-10T18:41:46.525308+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:48.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:47 smithi151 bash[22565]: audit 2024-09-10T18:41:46.505894+0000 mon.b (mon.1) 187 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:48.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:47 smithi151 bash[22565]: audit 2024-09-10T18:41:46.521897+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:48.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:47 smithi151 bash[22565]: audit 2024-09-10T18:41:46.522187+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:48.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:47 smithi151 bash[22565]: cluster 2024-09-10T18:41:46.522457+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:41:48.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:47 smithi151 bash[22565]: audit 2024-09-10T18:41:46.523380+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:48.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:47 smithi151 bash[22565]: audit 2024-09-10T18:41:46.524587+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:48.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:47 smithi151 bash[22565]: audit 2024-09-10T18:41:46.525308+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:49.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: cluster 2024-09-10T18:41:47.895036+0000 mgr.y (mgr.24512) 307 : cluster [DBG] pgmap v409: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: audit 2024-09-10T18:41:47.938454+0000 mon.b (mon.1) 188 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]: dispatch 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: audit 2024-09-10T18:41:47.954002+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]': finished 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: cluster 2024-09-10T18:41:47.954091+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: audit 2024-09-10T18:41:47.967771+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]: dispatch 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: cluster 2024-09-10T18:41:48.518876+0000 mon.a (mon.0) 2114 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: audit 2024-09-10T18:41:48.534230+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: audit 2024-09-10T18:41:48.534391+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]': finished 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: cluster 2024-09-10T18:41:48.535408+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: audit 2024-09-10T18:41:48.566245+0000 mon.b (mon.1) 189 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[18597]: audit 2024-09-10T18:41:48.584413+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: cluster 2024-09-10T18:41:47.895036+0000 mgr.y (mgr.24512) 307 : cluster [DBG] pgmap v409: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: audit 2024-09-10T18:41:47.938454+0000 mon.b (mon.1) 188 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]: dispatch 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: audit 2024-09-10T18:41:47.954002+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]': finished 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: cluster 2024-09-10T18:41:47.954091+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: audit 2024-09-10T18:41:47.967771+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]: dispatch 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: cluster 2024-09-10T18:41:48.518876+0000 mon.a (mon.0) 2114 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: audit 2024-09-10T18:41:48.534230+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: audit 2024-09-10T18:41:48.534391+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]': finished 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: cluster 2024-09-10T18:41:48.535408+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: audit 2024-09-10T18:41:48.566245+0000 mon.b (mon.1) 189 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:49.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:48 smithi100 bash[31607]: audit 2024-09-10T18:41:48.584413+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: cluster 2024-09-10T18:41:47.895036+0000 mgr.y (mgr.24512) 307 : cluster [DBG] pgmap v409: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: audit 2024-09-10T18:41:47.938454+0000 mon.b (mon.1) 188 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]: dispatch 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: audit 2024-09-10T18:41:47.954002+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-37"}]': finished 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: cluster 2024-09-10T18:41:47.954091+0000 mon.a (mon.0) 2112 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: audit 2024-09-10T18:41:47.967771+0000 mon.a (mon.0) 2113 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]: dispatch 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: cluster 2024-09-10T18:41:48.518876+0000 mon.a (mon.0) 2114 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: audit 2024-09-10T18:41:48.534230+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-72601-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: audit 2024-09-10T18:41:48.534391+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-37", "mode": "writeback"}]': finished 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: cluster 2024-09-10T18:41:48.535408+0000 mon.a (mon.0) 2117 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:41:49.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: audit 2024-09-10T18:41:48.566245+0000 mon.b (mon.1) 189 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:49.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:48 smithi151 bash[22565]: audit 2024-09-10T18:41:48.584413+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:50.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: audit 2024-09-10T18:41:49.524896+0000 mon.b (mon.1) 190 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:50.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: audit 2024-09-10T18:41:49.537758+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:50.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: cluster 2024-09-10T18:41:49.537857+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: audit 2024-09-10T18:41:49.543320+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: cluster 2024-09-10T18:41:49.895893+0000 mgr.y (mgr.24512) 308 : cluster [DBG] pgmap v412: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: cluster 2024-09-10T18:41:50.342460+0000 mon.a (mon.0) 2122 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: cluster 2024-09-10T18:41:50.343468+0000 mon.a (mon.0) 2123 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: audit 2024-09-10T18:41:50.353178+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]': finished 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: cluster 2024-09-10T18:41:50.353484+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: audit 2024-09-10T18:41:50.355725+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[31607]: audit 2024-09-10T18:41:50.356503+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: audit 2024-09-10T18:41:49.524896+0000 mon.b (mon.1) 190 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: audit 2024-09-10T18:41:49.537758+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: cluster 2024-09-10T18:41:49.537857+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: audit 2024-09-10T18:41:49.543320+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: cluster 2024-09-10T18:41:49.895893+0000 mgr.y (mgr.24512) 308 : cluster [DBG] pgmap v412: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: cluster 2024-09-10T18:41:50.342460+0000 mon.a (mon.0) 2122 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: cluster 2024-09-10T18:41:50.343468+0000 mon.a (mon.0) 2123 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: audit 2024-09-10T18:41:50.353178+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]': finished 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: cluster 2024-09-10T18:41:50.353484+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: audit 2024-09-10T18:41:50.355725+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:50.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:50 smithi100 bash[18597]: audit 2024-09-10T18:41:50.356503+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: audit 2024-09-10T18:41:49.524896+0000 mon.b (mon.1) 190 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: audit 2024-09-10T18:41:49.537758+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: cluster 2024-09-10T18:41:49.537857+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: audit 2024-09-10T18:41:49.543320+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]: dispatch 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: cluster 2024-09-10T18:41:49.895893+0000 mgr.y (mgr.24512) 308 : cluster [DBG] pgmap v412: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: cluster 2024-09-10T18:41:50.342460+0000 mon.a (mon.0) 2122 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: cluster 2024-09-10T18:41:50.343468+0000 mon.a (mon.0) 2123 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:50.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: audit 2024-09-10T18:41:50.353178+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-37"}]': finished 2024-09-10T18:41:50.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: cluster 2024-09-10T18:41:50.353484+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:41:50.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: audit 2024-09-10T18:41:50.355725+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:50.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:50 smithi151 bash[22565]: audit 2024-09-10T18:41:50.356503+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:41:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:51] "GET /metrics HTTP/1.1" 200 228463 "" "Prometheus/2.43.0" 2024-09-10T18:41:52.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:41:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:41:52.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:52 smithi151 bash[22565]: audit 2024-09-10T18:41:51.356293+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:52.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:52 smithi151 bash[22565]: cluster 2024-09-10T18:41:51.356429+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:41:52.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:52 smithi151 bash[22565]: audit 2024-09-10T18:41:51.358752+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:52.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:52 smithi151 bash[22565]: audit 2024-09-10T18:41:51.786292+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:52.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:52 smithi151 bash[22565]: cluster 2024-09-10T18:41:51.896744+0000 mgr.y (mgr.24512) 309 : cluster [DBG] pgmap v415: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:41:52.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[18597]: audit 2024-09-10T18:41:51.356293+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:52.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[18597]: cluster 2024-09-10T18:41:51.356429+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:41:52.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[18597]: audit 2024-09-10T18:41:51.358752+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:52.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[18597]: audit 2024-09-10T18:41:51.786292+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[18597]: cluster 2024-09-10T18:41:51.896744+0000 mgr.y (mgr.24512) 309 : cluster [DBG] pgmap v415: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:41:52.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[31607]: audit 2024-09-10T18:41:51.356293+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:52.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[31607]: cluster 2024-09-10T18:41:51.356429+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:41:52.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[31607]: audit 2024-09-10T18:41:51.358752+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.100:0/3033086646' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:52.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[31607]: audit 2024-09-10T18:41:51.786292+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]: dispatch 2024-09-10T18:41:52.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:52 smithi100 bash[31607]: cluster 2024-09-10T18:41:51.896744+0000 mgr.y (mgr.24512) 309 : cluster [DBG] pgmap v415: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: audit 2024-09-10T18:41:52.269476+0000 mgr.y (mgr.24512) 310 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: audit 2024-09-10T18:41:52.353196+0000 mon.b (mon.1) 191 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: audit 2024-09-10T18:41:52.366964+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: cluster 2024-09-10T18:41:52.367185+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: audit 2024-09-10T18:41:52.374191+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: audit 2024-09-10T18:41:52.395671+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: audit 2024-09-10T18:41:52.396652+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:53.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:53 smithi151 bash[22565]: audit 2024-09-10T18:41:52.397138+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-72601-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:53.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: audit 2024-09-10T18:41:52.269476+0000 mgr.y (mgr.24512) 310 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: audit 2024-09-10T18:41:52.353196+0000 mon.b (mon.1) 191 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: audit 2024-09-10T18:41:52.366964+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: cluster 2024-09-10T18:41:52.367185+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: audit 2024-09-10T18:41:52.374191+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: audit 2024-09-10T18:41:52.395671+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: audit 2024-09-10T18:41:52.396652+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[18597]: audit 2024-09-10T18:41:52.397138+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-72601-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: audit 2024-09-10T18:41:52.269476+0000 mgr.y (mgr.24512) 310 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: audit 2024-09-10T18:41:52.353196+0000 mon.b (mon.1) 191 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: audit 2024-09-10T18:41:52.366964+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-72601-53"}]': finished 2024-09-10T18:41:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: cluster 2024-09-10T18:41:52.367185+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:41:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: audit 2024-09-10T18:41:52.374191+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: audit 2024-09-10T18:41:52.395671+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: audit 2024-09-10T18:41:52.396652+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:53.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:53 smithi100 bash[31607]: audit 2024-09-10T18:41:52.397138+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-72601-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:54.805 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:41:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: audit 2024-09-10T18:41:53.364591+0000 mon.b (mon.1) 192 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: audit 2024-09-10T18:41:53.377254+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: audit 2024-09-10T18:41:53.377572+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-72601-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: cluster 2024-09-10T18:41:53.377679+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: audit 2024-09-10T18:41:53.380412+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-72601-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: audit 2024-09-10T18:41:53.383990+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: cluster 2024-09-10T18:41:53.897638+0000 mgr.y (mgr.24512) 311 : cluster [DBG] pgmap v418: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: audit 2024-09-10T18:41:54.334444+0000 mon.a (mon.0) 2141 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:55.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:54 smithi151 bash[22565]: audit 2024-09-10T18:41:54.337218+0000 mon.c (mon.2) 375 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:55.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: audit 2024-09-10T18:41:53.364591+0000 mon.b (mon.1) 192 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: audit 2024-09-10T18:41:53.377254+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: audit 2024-09-10T18:41:53.377572+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-72601-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: cluster 2024-09-10T18:41:53.377679+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: audit 2024-09-10T18:41:53.380412+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-72601-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: audit 2024-09-10T18:41:53.383990+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: cluster 2024-09-10T18:41:53.897638+0000 mgr.y (mgr.24512) 311 : cluster [DBG] pgmap v418: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: audit 2024-09-10T18:41:54.334444+0000 mon.a (mon.0) 2141 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:55.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[18597]: audit 2024-09-10T18:41:54.337218+0000 mon.c (mon.2) 375 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:55.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: audit 2024-09-10T18:41:53.364591+0000 mon.b (mon.1) 192 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:55.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: audit 2024-09-10T18:41:53.377254+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:55.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: audit 2024-09-10T18:41:53.377572+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-72601-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:55.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: cluster 2024-09-10T18:41:53.377679+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:41:55.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: audit 2024-09-10T18:41:53.380412+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-72601-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:55.298 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: audit 2024-09-10T18:41:53.383990+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:55.299 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: cluster 2024-09-10T18:41:53.897638+0000 mgr.y (mgr.24512) 311 : cluster [DBG] pgmap v418: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:41:55.299 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: audit 2024-09-10T18:41:54.334444+0000 mon.a (mon.0) 2141 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:41:55.299 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:54 smithi100 bash[31607]: audit 2024-09-10T18:41:54.337218+0000 mon.c (mon.2) 375 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: audit 2024-09-10T18:41:54.784666+0000 mon.b (mon.1) 193 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: audit 2024-09-10T18:41:54.800910+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: cluster 2024-09-10T18:41:54.801110+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: audit 2024-09-10T18:41:54.803802+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: audit 2024-09-10T18:41:55.373064+0000 mon.b (mon.1) 194 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]: dispatch 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: audit 2024-09-10T18:41:55.387492+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-72601-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: audit 2024-09-10T18:41:55.387772+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]': finished 2024-09-10T18:41:56.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: cluster 2024-09-10T18:41:55.387985+0000 mon.a (mon.0) 2147 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:41:56.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:55 smithi151 bash[22565]: audit 2024-09-10T18:41:55.391442+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]: dispatch 2024-09-10T18:41:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: audit 2024-09-10T18:41:54.784666+0000 mon.b (mon.1) 193 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: audit 2024-09-10T18:41:54.800910+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: cluster 2024-09-10T18:41:54.801110+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:41:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: audit 2024-09-10T18:41:54.803802+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: audit 2024-09-10T18:41:55.373064+0000 mon.b (mon.1) 194 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]: dispatch 2024-09-10T18:41:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: audit 2024-09-10T18:41:55.387492+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-72601-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:41:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: audit 2024-09-10T18:41:55.387772+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]': finished 2024-09-10T18:41:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: cluster 2024-09-10T18:41:55.387985+0000 mon.a (mon.0) 2147 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:41:56.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[31607]: audit 2024-09-10T18:41:55.391442+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]: dispatch 2024-09-10T18:41:56.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: audit 2024-09-10T18:41:54.784666+0000 mon.b (mon.1) 193 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:56.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: audit 2024-09-10T18:41:54.800910+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:56.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: cluster 2024-09-10T18:41:54.801110+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:41:56.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: audit 2024-09-10T18:41:54.803802+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:56.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: audit 2024-09-10T18:41:55.373064+0000 mon.b (mon.1) 194 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]: dispatch 2024-09-10T18:41:56.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: audit 2024-09-10T18:41:55.387492+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-72601-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:41:56.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: audit 2024-09-10T18:41:55.387772+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-39"}]': finished 2024-09-10T18:41:56.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: cluster 2024-09-10T18:41:55.387985+0000 mon.a (mon.0) 2147 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:41:56.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:55 smithi100 bash[18597]: audit 2024-09-10T18:41:55.391442+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]: dispatch 2024-09-10T18:41:57.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:56 smithi151 bash[22565]: cluster 2024-09-10T18:41:55.898478+0000 mgr.y (mgr.24512) 312 : cluster [DBG] pgmap v421: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:57.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:56 smithi151 bash[22565]: cluster 2024-09-10T18:41:56.379274+0000 mon.a (mon.0) 2149 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:57.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:56 smithi151 bash[22565]: audit 2024-09-10T18:41:56.390495+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]': finished 2024-09-10T18:41:57.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:56 smithi151 bash[22565]: cluster 2024-09-10T18:41:56.390696+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:41:57.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:56 smithi151 bash[22565]: audit 2024-09-10T18:41:56.505137+0000 mon.b (mon.1) 195 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:57.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:56 smithi151 bash[22565]: audit 2024-09-10T18:41:56.523531+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:57.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[31607]: cluster 2024-09-10T18:41:55.898478+0000 mgr.y (mgr.24512) 312 : cluster [DBG] pgmap v421: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:57.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[31607]: cluster 2024-09-10T18:41:56.379274+0000 mon.a (mon.0) 2149 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:57.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[31607]: audit 2024-09-10T18:41:56.390495+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]': finished 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[31607]: cluster 2024-09-10T18:41:56.390696+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[31607]: audit 2024-09-10T18:41:56.505137+0000 mon.b (mon.1) 195 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[31607]: audit 2024-09-10T18:41:56.523531+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[18597]: cluster 2024-09-10T18:41:55.898478+0000 mgr.y (mgr.24512) 312 : cluster [DBG] pgmap v421: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[18597]: cluster 2024-09-10T18:41:56.379274+0000 mon.a (mon.0) 2149 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[18597]: audit 2024-09-10T18:41:56.390495+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-39", "mode": "writeback"}]': finished 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[18597]: cluster 2024-09-10T18:41:56.390696+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[18597]: audit 2024-09-10T18:41:56.505137+0000 mon.b (mon.1) 195 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:56 smithi100 bash[18597]: audit 2024-09-10T18:41:56.523531+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:41:58.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:57 smithi151 bash[22565]: cluster 2024-09-10T18:41:56.806681+0000 mon.a (mon.0) 2153 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:58.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:57 smithi151 bash[22565]: audit 2024-09-10T18:41:57.375825+0000 mon.b (mon.1) 196 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:58.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:57 smithi151 bash[22565]: audit 2024-09-10T18:41:57.391733+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:58.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:57 smithi151 bash[22565]: cluster 2024-09-10T18:41:57.391930+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:41:58.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:57 smithi151 bash[22565]: audit 2024-09-10T18:41:57.394535+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:58.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:57 smithi151 bash[22565]: audit 2024-09-10T18:41:57.394839+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:58.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[31607]: cluster 2024-09-10T18:41:56.806681+0000 mon.a (mon.0) 2153 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:58.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[31607]: audit 2024-09-10T18:41:57.375825+0000 mon.b (mon.1) 196 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:58.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[31607]: audit 2024-09-10T18:41:57.391733+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[31607]: cluster 2024-09-10T18:41:57.391930+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[31607]: audit 2024-09-10T18:41:57.394535+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[31607]: audit 2024-09-10T18:41:57.394839+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[18597]: cluster 2024-09-10T18:41:56.806681+0000 mon.a (mon.0) 2153 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[18597]: audit 2024-09-10T18:41:57.375825+0000 mon.b (mon.1) 196 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[18597]: audit 2024-09-10T18:41:57.391733+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[18597]: cluster 2024-09-10T18:41:57.391930+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[18597]: audit 2024-09-10T18:41:57.394535+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:57 smithi100 bash[18597]: audit 2024-09-10T18:41:57.394839+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]: dispatch 2024-09-10T18:41:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:58 smithi151 bash[22565]: cluster 2024-09-10T18:41:57.899298+0000 mgr.y (mgr.24512) 313 : cluster [DBG] pgmap v424: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:58 smithi151 bash[22565]: cluster 2024-09-10T18:41:58.384714+0000 mon.a (mon.0) 2158 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:58 smithi151 bash[22565]: audit 2024-09-10T18:41:58.393540+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:41:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:58 smithi151 bash[22565]: audit 2024-09-10T18:41:58.393660+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]': finished 2024-09-10T18:41:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:58 smithi151 bash[22565]: cluster 2024-09-10T18:41:58.393754+0000 mon.a (mon.0) 2161 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:41:59.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:41:58 smithi151 bash[22565]: audit 2024-09-10T18:41:58.396888+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[31607]: cluster 2024-09-10T18:41:57.899298+0000 mgr.y (mgr.24512) 313 : cluster [DBG] pgmap v424: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[31607]: cluster 2024-09-10T18:41:58.384714+0000 mon.a (mon.0) 2158 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[31607]: audit 2024-09-10T18:41:58.393540+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:41:59.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[31607]: audit 2024-09-10T18:41:58.393660+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]': finished 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[31607]: cluster 2024-09-10T18:41:58.393754+0000 mon.a (mon.0) 2161 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[31607]: audit 2024-09-10T18:41:58.396888+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[18597]: cluster 2024-09-10T18:41:57.899298+0000 mgr.y (mgr.24512) 313 : cluster [DBG] pgmap v424: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[18597]: cluster 2024-09-10T18:41:58.384714+0000 mon.a (mon.0) 2158 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[18597]: audit 2024-09-10T18:41:58.393540+0000 mon.a (mon.0) 2159 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[18597]: audit 2024-09-10T18:41:58.393660+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-39"}]': finished 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[18597]: cluster 2024-09-10T18:41:58.393754+0000 mon.a (mon.0) 2161 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:41:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:41:58 smithi100 bash[18597]: audit 2024-09-10T18:41:58.396888+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]: dispatch 2024-09-10T18:41:59.399 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-10T18:41:59.399 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-10T18:41:59.399 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-10T18:41:59.399 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: seed 72764 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (41 ms) 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (14499 ms) 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (6 ms) 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (14546 ms total) 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7988 ms) 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8217 ms) 2024-09-10T18:41:59.400 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9998 ms) 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [4] 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [5,4] 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [6,5,4] 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting some heads 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 7 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: waiting for scrubs... 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: done waiting 2024-09-10T18:41:59.401 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46269 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9774 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8436 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (7633 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8053 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10047 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (9274 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (9616 ms) 2024-09-10T18:41:59.402 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (13989 ms) 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7970 ms) 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8488 ms) 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (12669 ms) 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7851 ms) 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-10T18:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8042 ms) 2024-09-10T18:42:00.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: audit 2024-09-10T18:41:59.396500+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:42:00.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: cluster 2024-09-10T18:41:59.396678+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:42:00.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: audit 2024-09-10T18:41:59.422510+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: audit 2024-09-10T18:41:59.826083+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: audit 2024-09-10T18:41:59.827075+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: audit 2024-09-10T18:41:59.827569+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: audit 2024-09-10T18:41:59.828364+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: audit 2024-09-10T18:41:59.828855+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:00 smithi151 bash[22565]: cluster 2024-09-10T18:41:59.900187+0000 mgr.y (mgr.24512) 314 : cluster [DBG] pgmap v427: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:00.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: audit 2024-09-10T18:41:59.396500+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:42:00.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: cluster 2024-09-10T18:41:59.396678+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: audit 2024-09-10T18:41:59.422510+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: audit 2024-09-10T18:41:59.826083+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: audit 2024-09-10T18:41:59.827075+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: audit 2024-09-10T18:41:59.827569+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: audit 2024-09-10T18:41:59.828364+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: audit 2024-09-10T18:41:59.828855+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[31607]: cluster 2024-09-10T18:41:59.900187+0000 mgr.y (mgr.24512) 314 : cluster [DBG] pgmap v427: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: audit 2024-09-10T18:41:59.396500+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? 172.21.15.100:0/2570893966' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-72601-54"}]': finished 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: cluster 2024-09-10T18:41:59.396678+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:42:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: audit 2024-09-10T18:41:59.422510+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: audit 2024-09-10T18:41:59.826083+0000 mon.a (mon.0) 2165 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: audit 2024-09-10T18:41:59.827075+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: audit 2024-09-10T18:41:59.827569+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: audit 2024-09-10T18:41:59.828364+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: audit 2024-09-10T18:41:59.828855+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:00 smithi100 bash[18597]: cluster 2024-09-10T18:41:59.900187+0000 mgr.y (mgr.24512) 314 : cluster [DBG] pgmap v427: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:01] "GET /metrics HTTP/1.1" 200 228466 "" "Prometheus/2.43.0" 2024-09-10T18:42:02.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:01 smithi151 bash[22565]: audit 2024-09-10T18:42:00.417791+0000 mon.b (mon.1) 197 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:02.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:01 smithi151 bash[22565]: audit 2024-09-10T18:42:00.427214+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:02.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:01 smithi151 bash[22565]: cluster 2024-09-10T18:42:00.427337+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:42:02.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:01 smithi151 bash[22565]: audit 2024-09-10T18:42:00.429365+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:02.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:01 smithi151 bash[22565]: audit 2024-09-10T18:42:00.433941+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:02.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:01 smithi151 bash[22565]: audit 2024-09-10T18:42:00.436214+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:02.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[18597]: audit 2024-09-10T18:42:00.417791+0000 mon.b (mon.1) 197 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:02.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[18597]: audit 2024-09-10T18:42:00.427214+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:02.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[18597]: cluster 2024-09-10T18:42:00.427337+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:42:02.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[18597]: audit 2024-09-10T18:42:00.429365+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[18597]: audit 2024-09-10T18:42:00.433941+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[18597]: audit 2024-09-10T18:42:00.436214+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[31607]: audit 2024-09-10T18:42:00.417791+0000 mon.b (mon.1) 197 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[31607]: audit 2024-09-10T18:42:00.427214+0000 mon.a (mon.0) 2168 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-72601-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[31607]: cluster 2024-09-10T18:42:00.427337+0000 mon.a (mon.0) 2169 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[31607]: audit 2024-09-10T18:42:00.429365+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[31607]: audit 2024-09-10T18:42:00.433941+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:02.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:01 smithi100 bash[31607]: audit 2024-09-10T18:42:00.436214+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:02.680 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:42:03.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: audit 2024-09-10T18:42:01.852912+0000 mon.b (mon.1) 198 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:03.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: audit 2024-09-10T18:42:01.858141+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:03.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: cluster 2024-09-10T18:42:01.858237+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:42:03.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: audit 2024-09-10T18:42:01.872687+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:03.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: cluster 2024-09-10T18:42:01.900991+0000 mgr.y (mgr.24512) 315 : cluster [DBG] pgmap v430: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:42:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: audit 2024-09-10T18:42:02.416918+0000 mon.b (mon.1) 199 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: audit 2024-09-10T18:42:02.433353+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: audit 2024-09-10T18:42:02.433494+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: cluster 2024-09-10T18:42:02.433592+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:42:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:02 smithi151 bash[22565]: audit 2024-09-10T18:42:02.436946+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:03.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: audit 2024-09-10T18:42:01.852912+0000 mon.b (mon.1) 198 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:03.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: audit 2024-09-10T18:42:01.858141+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:03.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: cluster 2024-09-10T18:42:01.858237+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: audit 2024-09-10T18:42:01.872687+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: cluster 2024-09-10T18:42:01.900991+0000 mgr.y (mgr.24512) 315 : cluster [DBG] pgmap v430: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: audit 2024-09-10T18:42:02.416918+0000 mon.b (mon.1) 199 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: audit 2024-09-10T18:42:02.433353+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: audit 2024-09-10T18:42:02.433494+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: cluster 2024-09-10T18:42:02.433592+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[31607]: audit 2024-09-10T18:42:02.436946+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: audit 2024-09-10T18:42:01.852912+0000 mon.b (mon.1) 198 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: audit 2024-09-10T18:42:01.858141+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: cluster 2024-09-10T18:42:01.858237+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:42:03.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: audit 2024-09-10T18:42:01.872687+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:03.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: cluster 2024-09-10T18:42:01.900991+0000 mgr.y (mgr.24512) 315 : cluster [DBG] pgmap v430: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:42:03.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: audit 2024-09-10T18:42:02.416918+0000 mon.b (mon.1) 199 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:03.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: audit 2024-09-10T18:42:02.433353+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-72601-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:03.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: audit 2024-09-10T18:42:02.433494+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:03.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: cluster 2024-09-10T18:42:02.433592+0000 mon.a (mon.0) 2177 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:42:03.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:02 smithi100 bash[18597]: audit 2024-09-10T18:42:02.436946+0000 mon.a (mon.0) 2178 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:04.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:03 smithi151 bash[22565]: audit 2024-09-10T18:42:02.281059+0000 mgr.y (mgr.24512) 316 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:04.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:03 smithi151 bash[22565]: audit 2024-09-10T18:42:03.421840+0000 mon.b (mon.1) 200 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:04.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:03 smithi151 bash[22565]: audit 2024-09-10T18:42:03.437549+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:42:04.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:03 smithi151 bash[22565]: cluster 2024-09-10T18:42:03.437800+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:42:04.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:03 smithi151 bash[22565]: audit 2024-09-10T18:42:03.440097+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[18597]: audit 2024-09-10T18:42:02.281059+0000 mgr.y (mgr.24512) 316 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[18597]: audit 2024-09-10T18:42:03.421840+0000 mon.b (mon.1) 200 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[18597]: audit 2024-09-10T18:42:03.437549+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:42:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[18597]: cluster 2024-09-10T18:42:03.437800+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:42:04.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[18597]: audit 2024-09-10T18:42:03.440097+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[31607]: audit 2024-09-10T18:42:02.281059+0000 mgr.y (mgr.24512) 316 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[31607]: audit 2024-09-10T18:42:03.421840+0000 mon.b (mon.1) 200 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[31607]: audit 2024-09-10T18:42:03.437549+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:42:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[31607]: cluster 2024-09-10T18:42:03.437800+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:42:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:03 smithi100 bash[31607]: audit 2024-09-10T18:42:03.440097+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:04.885 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:05.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[22565]: cluster 2024-09-10T18:42:03.901792+0000 mgr.y (mgr.24512) 317 : cluster [DBG] pgmap v433: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:42:05.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[22565]: audit 2024-09-10T18:42:04.421825+0000 mon.b (mon.1) 201 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:42:05.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[22565]: audit 2024-09-10T18:42:04.437164+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:05.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[22565]: cluster 2024-09-10T18:42:04.437523+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:42:05.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[22565]: audit 2024-09-10T18:42:04.440656+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:05.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[22565]: audit 2024-09-10T18:42:04.443062+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:42:05.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:04 smithi151 bash[22565]: audit 2024-09-10T18:42:04.443339+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[18597]: cluster 2024-09-10T18:42:03.901792+0000 mgr.y (mgr.24512) 317 : cluster [DBG] pgmap v433: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:42:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[18597]: audit 2024-09-10T18:42:04.421825+0000 mon.b (mon.1) 201 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:42:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[18597]: audit 2024-09-10T18:42:04.437164+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[18597]: cluster 2024-09-10T18:42:04.437523+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[18597]: audit 2024-09-10T18:42:04.440656+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[18597]: audit 2024-09-10T18:42:04.443062+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[18597]: audit 2024-09-10T18:42:04.443339+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[31607]: cluster 2024-09-10T18:42:03.901792+0000 mgr.y (mgr.24512) 317 : cluster [DBG] pgmap v433: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[31607]: audit 2024-09-10T18:42:04.421825+0000 mon.b (mon.1) 201 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[31607]: audit 2024-09-10T18:42:04.437164+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[31607]: cluster 2024-09-10T18:42:04.437523+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[31607]: audit 2024-09-10T18:42:04.440656+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[31607]: audit 2024-09-10T18:42:04.443062+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:42:05.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:04 smithi100 bash[31607]: audit 2024-09-10T18:42:04.443339+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:05 smithi151 bash[22565]: cluster 2024-09-10T18:42:04.873506+0000 mon.a (mon.0) 2186 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:05 smithi151 bash[22565]: audit 2024-09-10T18:42:05.442280+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:42:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:05 smithi151 bash[22565]: audit 2024-09-10T18:42:05.442416+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:05 smithi151 bash[22565]: cluster 2024-09-10T18:42:05.442501+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:42:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:05 smithi151 bash[22565]: audit 2024-09-10T18:42:05.445253+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:06.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:05 smithi151 bash[22565]: audit 2024-09-10T18:42:05.445895+0000 mon.a (mon.0) 2190 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[18597]: cluster 2024-09-10T18:42:04.873506+0000 mon.a (mon.0) 2186 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[18597]: audit 2024-09-10T18:42:05.442280+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[18597]: audit 2024-09-10T18:42:05.442416+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[18597]: cluster 2024-09-10T18:42:05.442501+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[18597]: audit 2024-09-10T18:42:05.445253+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[18597]: audit 2024-09-10T18:42:05.445895+0000 mon.a (mon.0) 2190 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[31607]: cluster 2024-09-10T18:42:04.873506+0000 mon.a (mon.0) 2186 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[31607]: audit 2024-09-10T18:42:05.442280+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:42:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[31607]: audit 2024-09-10T18:42:05.442416+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[31607]: cluster 2024-09-10T18:42:05.442501+0000 mon.a (mon.0) 2189 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:42:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[31607]: audit 2024-09-10T18:42:05.445253+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.100:0/3739983037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:06.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:05 smithi100 bash[31607]: audit 2024-09-10T18:42:05.445895+0000 mon.a (mon.0) 2190 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]: dispatch 2024-09-10T18:42:07.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: cluster 2024-09-10T18:42:05.902312+0000 mgr.y (mgr.24512) 318 : cluster [DBG] pgmap v436: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:07.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.443185+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:07.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: cluster 2024-09-10T18:42:06.443363+0000 mon.a (mon.0) 2192 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.467847+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.469435+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.471540+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.471887+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.472706+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.473081+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.473605+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.473905+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.486379+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:06 smithi151 bash[22565]: audit 2024-09-10T18:42:06.487783+0000 mon.a (mon.0) 2197 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]: dispatch 2024-09-10T18:42:07.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: cluster 2024-09-10T18:42:05.902312+0000 mgr.y (mgr.24512) 318 : cluster [DBG] pgmap v436: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.443185+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: cluster 2024-09-10T18:42:06.443363+0000 mon.a (mon.0) 2192 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.467847+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.469435+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]: dispatch 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.471540+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.471887+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.472706+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.473081+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.473605+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.473905+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.486379+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[18597]: audit 2024-09-10T18:42:06.487783+0000 mon.a (mon.0) 2197 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: cluster 2024-09-10T18:42:05.902312+0000 mgr.y (mgr.24512) 318 : cluster [DBG] pgmap v436: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.443185+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-72601-55"}]': finished 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: cluster 2024-09-10T18:42:06.443363+0000 mon.a (mon.0) 2192 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.467847+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.469435+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.471540+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.471887+0000 mon.a (mon.0) 2193 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.472706+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.473081+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.473605+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.473905+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.486379+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:06 smithi100 bash[31607]: audit 2024-09-10T18:42:06.487783+0000 mon.a (mon.0) 2197 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]: dispatch 2024-09-10T18:42:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[18597]: audit 2024-09-10T18:42:07.450755+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[18597]: audit 2024-09-10T18:42:07.451099+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]': finished 2024-09-10T18:42:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[18597]: cluster 2024-09-10T18:42:07.451249+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:42:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[18597]: audit 2024-09-10T18:42:07.453974+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[18597]: audit 2024-09-10T18:42:07.454726+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[18597]: cluster 2024-09-10T18:42:07.903070+0000 mgr.y (mgr.24512) 319 : cluster [DBG] pgmap v439: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[31607]: audit 2024-09-10T18:42:07.450755+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[31607]: audit 2024-09-10T18:42:07.451099+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]': finished 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[31607]: cluster 2024-09-10T18:42:07.451249+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[31607]: audit 2024-09-10T18:42:07.453974+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[31607]: audit 2024-09-10T18:42:07.454726+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:08 smithi100 bash[31607]: cluster 2024-09-10T18:42:07.903070+0000 mgr.y (mgr.24512) 319 : cluster [DBG] pgmap v439: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:08.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:08 smithi151 bash[22565]: audit 2024-09-10T18:42:07.450755+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-72601-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:08.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:08 smithi151 bash[22565]: audit 2024-09-10T18:42:07.451099+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-41"}]': finished 2024-09-10T18:42:08.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:08 smithi151 bash[22565]: cluster 2024-09-10T18:42:07.451249+0000 mon.a (mon.0) 2200 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:42:08.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:08 smithi151 bash[22565]: audit 2024-09-10T18:42:07.453974+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:08.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:08 smithi151 bash[22565]: audit 2024-09-10T18:42:07.454726+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:08.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:08 smithi151 bash[22565]: cluster 2024-09-10T18:42:07.903070+0000 mgr.y (mgr.24512) 319 : cluster [DBG] pgmap v439: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:09 smithi100 bash[18597]: cluster 2024-09-10T18:42:08.886466+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:42:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:09 smithi100 bash[18597]: audit 2024-09-10T18:42:09.326928+0000 mon.c (mon.2) 386 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:09 smithi100 bash[31607]: cluster 2024-09-10T18:42:08.886466+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:42:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:09 smithi100 bash[31607]: audit 2024-09-10T18:42:09.326928+0000 mon.c (mon.2) 386 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:09.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:09 smithi151 bash[22565]: cluster 2024-09-10T18:42:08.886466+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:42:09.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:09 smithi151 bash[22565]: audit 2024-09-10T18:42:09.326928+0000 mon.c (mon.2) 386 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[18597]: audit 2024-09-10T18:42:09.456795+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[18597]: audit 2024-09-10T18:42:09.467266+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[18597]: cluster 2024-09-10T18:42:09.467388+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[18597]: audit 2024-09-10T18:42:09.476392+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[18597]: cluster 2024-09-10T18:42:09.904016+0000 mgr.y (mgr.24512) 320 : cluster [DBG] pgmap v442: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[18597]: cluster 2024-09-10T18:42:10.354847+0000 mon.a (mon.0) 2206 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[31607]: audit 2024-09-10T18:42:09.456795+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[31607]: audit 2024-09-10T18:42:09.467266+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[31607]: cluster 2024-09-10T18:42:09.467388+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:42:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[31607]: audit 2024-09-10T18:42:09.476392+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[31607]: cluster 2024-09-10T18:42:09.904016+0000 mgr.y (mgr.24512) 320 : cluster [DBG] pgmap v442: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:10 smithi100 bash[31607]: cluster 2024-09-10T18:42:10.354847+0000 mon.a (mon.0) 2206 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:10.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:10 smithi151 bash[22565]: audit 2024-09-10T18:42:09.456795+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:10 smithi151 bash[22565]: audit 2024-09-10T18:42:09.467266+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-72601-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:10.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:10 smithi151 bash[22565]: cluster 2024-09-10T18:42:09.467388+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:42:10.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:10 smithi151 bash[22565]: audit 2024-09-10T18:42:09.476392+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:10 smithi151 bash[22565]: cluster 2024-09-10T18:42:09.904016+0000 mgr.y (mgr.24512) 320 : cluster [DBG] pgmap v442: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:10.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:10 smithi151 bash[22565]: cluster 2024-09-10T18:42:10.354847+0000 mon.a (mon.0) 2206 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:11] "GET /metrics HTTP/1.1" 200 226064 "" "Prometheus/2.43.0" 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[18597]: audit 2024-09-10T18:42:10.460644+0000 mon.b (mon.1) 205 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-72764-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[18597]: audit 2024-09-10T18:42:10.468632+0000 mon.b (mon.1) 206 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[18597]: audit 2024-09-10T18:42:10.474659+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[18597]: cluster 2024-09-10T18:42:10.474931+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[18597]: audit 2024-09-10T18:42:10.487015+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[31607]: audit 2024-09-10T18:42:10.460644+0000 mon.b (mon.1) 205 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-72764-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[31607]: audit 2024-09-10T18:42:10.468632+0000 mon.b (mon.1) 206 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[31607]: audit 2024-09-10T18:42:10.474659+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[31607]: cluster 2024-09-10T18:42:10.474931+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:42:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:11 smithi100 bash[31607]: audit 2024-09-10T18:42:10.487015+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:11.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:11 smithi151 bash[22565]: audit 2024-09-10T18:42:10.460644+0000 mon.b (mon.1) 205 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-72764-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:42:11.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:11 smithi151 bash[22565]: audit 2024-09-10T18:42:10.468632+0000 mon.b (mon.1) 206 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:11.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:11 smithi151 bash[22565]: audit 2024-09-10T18:42:10.474659+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:11.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:11 smithi151 bash[22565]: cluster 2024-09-10T18:42:10.474931+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:42:11.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:11 smithi151 bash[22565]: audit 2024-09-10T18:42:10.487015+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:12.680 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:42:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[22565]: audit 2024-09-10T18:42:11.481626+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:42:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[22565]: audit 2024-09-10T18:42:11.498021+0000 mon.a (mon.0) 2210 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[22565]: cluster 2024-09-10T18:42:11.498222+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:42:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[22565]: audit 2024-09-10T18:42:11.499832+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[22565]: audit 2024-09-10T18:42:11.500187+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:42:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[22565]: audit 2024-09-10T18:42:11.500484+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:12 smithi151 bash[22565]: cluster 2024-09-10T18:42:11.904789+0000 mgr.y (mgr.24512) 321 : cluster [DBG] pgmap v445: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[18597]: audit 2024-09-10T18:42:11.481626+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:42:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[18597]: audit 2024-09-10T18:42:11.498021+0000 mon.a (mon.0) 2210 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[18597]: cluster 2024-09-10T18:42:11.498222+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[18597]: audit 2024-09-10T18:42:11.499832+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[18597]: audit 2024-09-10T18:42:11.500187+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[18597]: audit 2024-09-10T18:42:11.500484+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[18597]: cluster 2024-09-10T18:42:11.904789+0000 mgr.y (mgr.24512) 321 : cluster [DBG] pgmap v445: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[31607]: audit 2024-09-10T18:42:11.481626+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[31607]: audit 2024-09-10T18:42:11.498021+0000 mon.a (mon.0) 2210 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[31607]: cluster 2024-09-10T18:42:11.498222+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[31607]: audit 2024-09-10T18:42:11.499832+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[31607]: audit 2024-09-10T18:42:11.500187+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:42:12.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[31607]: audit 2024-09-10T18:42:11.500484+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:12.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:12 smithi100 bash[31607]: cluster 2024-09-10T18:42:11.904789+0000 mgr.y (mgr.24512) 321 : cluster [DBG] pgmap v445: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:13.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRea1,1992,1993,1994,1995,1996,1997,1998,1999,2000,2001,2002,2003,2004,2005,2006,2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018,2019,2020,2021,2022,2023,2024,2025,2026,2027,2028,2029,2030,2031,2032,2033,2034,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047,2048,2049,2050,2051,2052,2053,2054,2055,2056,2057,2058,2059,2060,2061,2062,2063,2064,2065,2066,2067,2068,2069,2070,2071,2072,2073,2074,2075,2076,2077,2078,2079,2080,2081,2082,2083,2084,2085,2086,2087,2088,2089,2090,2091,2092,2093,2094,2095,2096,2097,2098,2099,2100,2101,2102,2103,2104,2105,2106,2107,2108,2109,2110,2111,2112,2113,2114,2115,2116,2117,2118,2119,2120,2121,2122,2123,2124,2125,2126,2127,2128,2129,2130,2131,2132,2133,2134,2135,2136,2137,2138,2139,2140,2141,2142,2143,2144,2145,2146,2147,2148,2149,2150,2151,2152,2153,2154,2155,2156,2157,2158,2159,2160,2161,2162,2163,2164,2165,2166,2167,2168,2169,2170,2171,2172,2173,2174,2175,2176,2177,2178,2179,2180,2181,2182,2183,2184,2185,2186,2187,2188,2189,2190,2191,2192,2193,2194,2195,2196,2197,2198,2199,2200,2201,2202,2203,2204,2205,2206,2207,2208,2209,2210,2211,2212,2213,2214,2215,2216,2217,2218,2219,2220,2221,2222 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: max_success 1197, min_failed 1198 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (5037 ms) 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3007 ms) 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (133122 ms total) 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3038 ms) 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9034 ms) 2024-09-10T18:42:13.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14878 ms) 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3452 ms) 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30402 ms total) 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (2575 ms) 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (2575 ms total) 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13992 ms) 2024-09-10T18:42:13.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6961 ms) 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7045 ms) 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7053 ms) 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3022 ms) 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7057 ms) 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6852 ms) 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7026 ms) 2024-09-10T18:42:13.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-10T18:42:13.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7048 ms) 2024-09-10T18:42:13.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-10T18:42:13.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: audit 2024-09-10T18:42:12.292726+0000 mgr.y (mgr.24512) 322 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: audit 2024-09-10T18:42:12.484657+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: audit 2024-09-10T18:42:12.500927+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: audit 2024-09-10T18:42:12.501117+0000 mon.a (mon.0) 2215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: cluster 2024-09-10T18:42:12.501228+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: audit 2024-09-10T18:42:12.503574+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: audit 2024-09-10T18:42:12.503575+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[18597]: audit 2024-09-10T18:42:12.510790+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: audit 2024-09-10T18:42:12.292726+0000 mgr.y (mgr.24512) 322 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: audit 2024-09-10T18:42:12.484657+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: audit 2024-09-10T18:42:12.500927+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:42:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: audit 2024-09-10T18:42:12.501117+0000 mon.a (mon.0) 2215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: cluster 2024-09-10T18:42:12.501228+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:42:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: audit 2024-09-10T18:42:12.503574+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: audit 2024-09-10T18:42:12.503575+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:13 smithi100 bash[31607]: audit 2024-09-10T18:42:12.510790+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:13.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: audit 2024-09-10T18:42:12.292726+0000 mgr.y (mgr.24512) 322 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:13.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: audit 2024-09-10T18:42:12.484657+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:13.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: audit 2024-09-10T18:42:12.500927+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:42:13.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: audit 2024-09-10T18:42:12.501117+0000 mon.a (mon.0) 2215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:13.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: cluster 2024-09-10T18:42:12.501228+0000 mon.a (mon.0) 2216 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:42:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: audit 2024-09-10T18:42:12.503574+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: audit 2024-09-10T18:42:12.503575+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.100:0/1281943824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:13 smithi151 bash[22565]: audit 2024-09-10T18:42:12.510790+0000 mon.a (mon.0) 2218 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]: dispatch 2024-09-10T18:42:14.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.490091+0000 mon.b (mon.1) 209 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:42:14.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.505013+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.505295+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: cluster 2024-09-10T18:42:13.505453+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.515589+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.535008+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.535450+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.536412+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.536834+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.537572+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: audit 2024-09-10T18:42:13.537998+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[18597]: cluster 2024-09-10T18:42:13.905615+0000 mgr.y (mgr.24512) 323 : cluster [DBG] pgmap v448: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:14.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.490091+0000 mon.b (mon.1) 209 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.505013+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.505295+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: cluster 2024-09-10T18:42:13.505453+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.515589+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.535008+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.535450+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.536412+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.536834+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.537572+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: audit 2024-09-10T18:42:13.537998+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:14 smithi100 bash[31607]: cluster 2024-09-10T18:42:13.905615+0000 mgr.y (mgr.24512) 323 : cluster [DBG] pgmap v448: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:14.930 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:14.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.490091+0000 mon.b (mon.1) 209 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:42:14.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.505013+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:14.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.505295+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-72601-56"}]': finished 2024-09-10T18:42:14.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: cluster 2024-09-10T18:42:13.505453+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.515589+0000 mon.a (mon.0) 2222 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.535008+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.535450+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.536412+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.536834+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.537572+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: audit 2024-09-10T18:42:13.537998+0000 mon.a (mon.0) 2225 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:14 smithi151 bash[22565]: cluster 2024-09-10T18:42:13.905615+0000 mgr.y (mgr.24512) 323 : cluster [DBG] pgmap v448: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: d 2024-09-10T18:42:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-10T18:42:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: ok, hit_set contains 192:602f83fe:::foo:head 2024-09-10T18:42:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9487 ms) 2024-09-10T18:42:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-10T18:42:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg_num = 32 2024-09-10T18:42:15.060 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 0 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 1 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 2 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 3 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 4 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 5 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 6 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 7 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 8 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 9 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 10 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 11 ls 1725993735,0 2024-09-10T18:42:15.061 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 12 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 13 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 14 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 15 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 16 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 17 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 18 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 19 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 20 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 21 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 22 ls 1725993735,0 2024-09-10T18:42:15.062 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 23 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 24 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 25 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 26 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 27 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 28 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 29 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 30 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 31 ls 1725993735,0 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg_num = 32 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6cac518f:::0:head 2024-09-10T18:42:15.063 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:02547ec2:::1:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f905c69b:::2:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:cfc208b3:::3:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d83876eb:::4:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b29083e3:::5:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c4fdafeb:::6:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:5c6b0b28:::7:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:bd63b0f1:::8:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:e960b815:::9:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:52ea6a34:::10:head 2024-09-10T18:42:15.064 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:89d3ae78:::11:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:de5d7c5f:::12:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:566253c9:::13:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:62a1935d:::14:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:863748b0:::15:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:3958e169:::16:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:4d4dabf9:::17:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:8391935d:::18:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:28883081:::19:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:69259c59:::20:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:4bdb80b7:::21:head 2024-09-10T18:42:15.065 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a11c5d71:::22:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:271af37b:::23:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:95b121be:::24:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:58d1031b:::25:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:0a050783:::26:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c709704c:::27:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:cbe56eaf:::28:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:86b4b162:::29:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:70d89383:::30:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:dd450c7c:::31:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6d5729b1:::32:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c388f3fb:::33:head 2024-09-10T18:42:15.066 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:56cfea31:::34:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:9dbc1bf7:::35:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:40b74ccd:::36:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:4d5aaf42:::37:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:920f362c:::38:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6cc53222:::39:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:9cad833f:::40:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:1ea84d41:::41:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c4480ef6:::42:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a694361e:::43:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d1bd33e9:::44:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:ddc2cd5d:::45:head 2024-09-10T18:42:15.067 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:2b782207:::46:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:7b187fca:::47:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:90ecdf6f:::48:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a5ed95fe:::49:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:ea0eaa55:::50:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f33ef17b:::51:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a0d1b2f6:::52:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:60c5229e:::53:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:edcbc575:::54:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:102cf253:::55:head 2024-09-10T18:42:15.068 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:efb7fb0b:::56:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:50d0a326:::57:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d4dc5daf:::58:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:3a130462:::59:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:ec87ed71:::60:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d5bc9454:::61:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:3ddfe313:::62:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:7c2816b9:::63:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:47e00e4d:::64:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c6410c18:::65:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b48ed237:::66:head 2024-09-10T18:42:15.069 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:cd63ad31:::67:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b179e92b:::68:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:0d9f741a:::69:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6d3352ae:::70:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c6d5c19e:::71:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:bc4729c3:::72:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:77e930b9:::73:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:0abeecfd:::74:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b7c37e15:::75:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b6378398:::76:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:02bd68de:::77:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:cc795d2d:::78:head 2024-09-10T18:42:15.070 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:630d4fea:::79:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:e0d29ef5:::80:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:fd6f13d2:::81:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:606461d5:::82:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:eadbdc43:::83:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:8761d0bb:::84:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:9ef0186f:::85:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:e0d41294:::86:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:961de695:::87:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:1423148f:::88:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:633a8fa2:::89:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a8653809:::90:head 2024-09-10T18:42:15.071 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:3dac8b33:::91:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:35aad435:::92:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f6dcc343:::93:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:dbbdad87:::94:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:1cb48ce0:::95:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:03cd461c:::96:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:17a4ea99:::97:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:9993c9a7:::98:head 2024-09-10T18:42:15.072 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6394211c:::99:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:94c7ae57:::100:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6fdee5bb:::101:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:9a477fd1:::102:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:eb850916:::103:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:affc56b9:::104:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b42dc814:::105:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f319f8f0:::106:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:9a40b9de:::107:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:8b524f28:::108:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:e3de589f:::109:head 2024-09-10T18:42:15.073 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:90f90a5b:::110:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a7b4f1d7:::111:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:af51766e:::112:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b6f90bd1:::113:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:e0261208:::114:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c9569ef7:::115:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:61bebe50:::116:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:fe93412b:::117:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d3d38bee:::118:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:3100ba0c:::119:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d0560ada:::120:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f0ea8b35:::121:head 2024-09-10T18:42:15.074 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:766f231a:::122:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a07a2582:::123:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:bd7c6b3a:::124:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:fb2ddaff:::125:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:4408e1fe:::126:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:ee1df7a7:::127:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c3002909:::128:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:4f48ffa9:::129:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:edf38733:::130:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c08425c0:::131:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:5f902d98:::132:head 2024-09-10T18:42:15.075 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:41ea2c93:::133:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:813cee13:::134:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:0131818d:::135:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:26ba5a85:::136:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:381b8a5a:::137:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:28797e47:::138:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:bfca7f22:::139:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:36807075:::140:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:80b03975:::141:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:5c15709b:::142:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f39ea15e:::143:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:ea992956:::144:head 2024-09-10T18:42:15.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:48887b1c:::145:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:9f24a9dd:::146:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:987f100b:::147:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d2dd3581:::148:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:7fed1808:::149:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c80b70e9:::150:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:85ed90f9:::151:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:36428b24:::152:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d044c34a:::153:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:7c18bf58:::154:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d1c21232:::155:head 2024-09-10T18:42:15.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a7a3c575:::156:head 2024-09-10T18:42:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:87da0633:::157:head 2024-09-10T18:42:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d5ac3822:::158:head 2024-09-10T18:42:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:3f20522d:::159:head 2024-09-10T18:42:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6ca26563:::160:head 2024-09-10T18:42:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:532ce135:::161:head 2024-09-10T18:42:15.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:c78863e6:::162:head 2024-09-10T18:42:16.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:14.518933+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:42:16.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:14.519196+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: cluster 2024-09-10T18:42:14.519410+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:14.521865+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:14.522619+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:15.041924+0000 mon.b (mon.1) 210 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-72764-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:15.361456+0000 mon.b (mon.1) 211 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:15.363633+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]: dispatch 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:15.380154+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[18597]: audit 2024-09-10T18:42:15.382439+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]: dispatch 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:14.518933+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:14.519196+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:16.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: cluster 2024-09-10T18:42:14.519410+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:42:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:14.521865+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:14.522619+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:15.041924+0000 mon.b (mon.1) 210 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-72764-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:42:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:15.361456+0000 mon.b (mon.1) 211 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:15.363633+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]: dispatch 2024-09-10T18:42:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:15.380154+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:16.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:15 smithi100 bash[31607]: audit 2024-09-10T18:42:15.382439+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]: dispatch 2024-09-10T18:42:16.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:14.518933+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:42:16.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:14.519196+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-72601-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:16.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: cluster 2024-09-10T18:42:14.519410+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:42:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:14.521865+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:14.522619+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:15.041924+0000 mon.b (mon.1) 210 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-72764-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:42:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:15.361456+0000 mon.b (mon.1) 211 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:15.363633+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]: dispatch 2024-09-10T18:42:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:15.380154+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:15 smithi151 bash[22565]: audit 2024-09-10T18:42:15.382439+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]: dispatch 2024-09-10T18:42:17.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[18597]: cluster 2024-09-10T18:42:15.906355+0000 mgr.y (mgr.24512) 324 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-09-10T18:42:17.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[18597]: audit 2024-09-10T18:42:15.964349+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]': finished 2024-09-10T18:42:17.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[18597]: cluster 2024-09-10T18:42:15.964541+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:42:17.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[18597]: audit 2024-09-10T18:42:16.533564+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:17.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[18597]: cluster 2024-09-10T18:42:16.533800+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:42:17.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[31607]: cluster 2024-09-10T18:42:15.906355+0000 mgr.y (mgr.24512) 324 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-09-10T18:42:17.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[31607]: audit 2024-09-10T18:42:15.964349+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]': finished 2024-09-10T18:42:17.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[31607]: cluster 2024-09-10T18:42:15.964541+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:42:17.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[31607]: audit 2024-09-10T18:42:16.533564+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:17.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:16 smithi100 bash[31607]: cluster 2024-09-10T18:42:16.533800+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:42:17.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:16 smithi151 bash[22565]: cluster 2024-09-10T18:42:15.906355+0000 mgr.y (mgr.24512) 324 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-09-10T18:42:17.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:16 smithi151 bash[22565]: audit 2024-09-10T18:42:15.964349+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-43"}]': finished 2024-09-10T18:42:17.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:16 smithi151 bash[22565]: cluster 2024-09-10T18:42:15.964541+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:42:17.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:16 smithi151 bash[22565]: audit 2024-09-10T18:42:16.533564+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-72601-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:17.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:16 smithi151 bash[22565]: cluster 2024-09-10T18:42:16.533800+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:42:18.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:18 smithi151 bash[22565]: audit 2024-09-10T18:42:17.529092+0000 mon.b (mon.1) 213 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:18.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:18 smithi151 bash[22565]: cluster 2024-09-10T18:42:17.541720+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:42:18.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:18 smithi151 bash[22565]: audit 2024-09-10T18:42:17.548446+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:18.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:18 smithi151 bash[22565]: cluster 2024-09-10T18:42:17.907307+0000 mgr.y (mgr.24512) 325 : cluster [DBG] pgmap v454: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[18597]: audit 2024-09-10T18:42:17.529092+0000 mon.b (mon.1) 213 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[18597]: cluster 2024-09-10T18:42:17.541720+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:42:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[18597]: audit 2024-09-10T18:42:17.548446+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[18597]: cluster 2024-09-10T18:42:17.907307+0000 mgr.y (mgr.24512) 325 : cluster [DBG] pgmap v454: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[31607]: audit 2024-09-10T18:42:17.529092+0000 mon.b (mon.1) 213 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[31607]: cluster 2024-09-10T18:42:17.541720+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:42:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[31607]: audit 2024-09-10T18:42:17.548446+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:18 smithi100 bash[31607]: cluster 2024-09-10T18:42:17.907307+0000 mgr.y (mgr.24512) 325 : cluster [DBG] pgmap v454: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:19.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: cluster 2024-09-10T18:42:18.533966+0000 mon.a (mon.0) 2238 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:19.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:18.536233+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:19.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:18.542463+0000 mon.a (mon.0) 2239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: cluster 2024-09-10T18:42:18.543033+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:18.545824+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:18.546873+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:18.564336+0000 mon.a (mon.0) 2242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:19.530414+0000 mon.b (mon.1) 215 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:19.546343+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:19.546621+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: cluster 2024-09-10T18:42:19.546860+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:19.549322+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:19.551024+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:19.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:19 smithi151 bash[22565]: audit 2024-09-10T18:42:19.551554+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: cluster 2024-09-10T18:42:18.533966+0000 mon.a (mon.0) 2238 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:18.536233+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:18.542463+0000 mon.a (mon.0) 2239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: cluster 2024-09-10T18:42:18.543033+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:18.545824+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:18.546873+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:18.564336+0000 mon.a (mon.0) 2242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:19.530414+0000 mon.b (mon.1) 215 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:19.546343+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:19.546621+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: cluster 2024-09-10T18:42:19.546860+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:19.549322+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:19.551024+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[18597]: audit 2024-09-10T18:42:19.551554+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: cluster 2024-09-10T18:42:18.533966+0000 mon.a (mon.0) 2238 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:18.536233+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:18.542463+0000 mon.a (mon.0) 2239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: cluster 2024-09-10T18:42:18.543033+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:42:20.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:18.545824+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:18.546873+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:18.564336+0000 mon.a (mon.0) 2242 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:19.530414+0000 mon.b (mon.1) 215 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:19.546343+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:19.546621+0000 mon.a (mon.0) 2244 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: cluster 2024-09-10T18:42:19.546860+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:19.549322+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:19.551024+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.100:0/949156398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:19 smithi100 bash[31607]: audit 2024-09-10T18:42:19.551554+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]: dispatch 2024-09-10T18:42:20.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:20 smithi151 bash[22565]: cluster 2024-09-10T18:42:19.908136+0000 mgr.y (mgr.24512) 326 : cluster [DBG] pgmap v457: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:20.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:20 smithi151 bash[22565]: audit 2024-09-10T18:42:20.532275+0000 mon.b (mon.1) 216 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:42:20.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:20 smithi151 bash[22565]: audit 2024-09-10T18:42:20.548535+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:42:20.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:20 smithi151 bash[22565]: audit 2024-09-10T18:42:20.548820+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:20.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:20 smithi151 bash[22565]: cluster 2024-09-10T18:42:20.548977+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:42:20.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:20 smithi151 bash[22565]: audit 2024-09-10T18:42:20.551083+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:42:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[18597]: cluster 2024-09-10T18:42:19.908136+0000 mgr.y (mgr.24512) 326 : cluster [DBG] pgmap v457: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:21.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[18597]: audit 2024-09-10T18:42:20.532275+0000 mon.b (mon.1) 216 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[18597]: audit 2024-09-10T18:42:20.548535+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[18597]: audit 2024-09-10T18:42:20.548820+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[18597]: cluster 2024-09-10T18:42:20.548977+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[18597]: audit 2024-09-10T18:42:20.551083+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[31607]: cluster 2024-09-10T18:42:19.908136+0000 mgr.y (mgr.24512) 326 : cluster [DBG] pgmap v457: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[31607]: audit 2024-09-10T18:42:20.532275+0000 mon.b (mon.1) 216 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[31607]: audit 2024-09-10T18:42:20.548535+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[31607]: audit 2024-09-10T18:42:20.548820+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-72601-57"}]': finished 2024-09-10T18:42:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[31607]: cluster 2024-09-10T18:42:20.548977+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:42:21.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:20 smithi100 bash[31607]: audit 2024-09-10T18:42:20.551083+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:42:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:21] "GET /metrics HTTP/1.1" 200 226064 "" "Prometheus/2.43.0" 2024-09-10T18:42:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:20.577215+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:20.577718+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:20.578300+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:20.578567+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:20.579103+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:20.579402+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:21.536254+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:21.552494+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:21.553016+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: cluster 2024-09-10T18:42:21.553115+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:21.554769+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:21.555631+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[18597]: audit 2024-09-10T18:42:21.556041+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:20.577215+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:20.577718+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:20.578300+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:20.578567+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:20.579103+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:20.579402+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:21.536254+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:21.552494+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:21.553016+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: cluster 2024-09-10T18:42:21.553115+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:42:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:21.554769+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:21.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:21.555631+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:21 smithi100 bash[31607]: audit 2024-09-10T18:42:21.556041+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:20.577215+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:20.577718+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:20.578300+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:20.578567+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:20.579103+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:20.579402+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:21.536254+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:21.552494+0000 mon.a (mon.0) 2255 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:21.553016+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-72601-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: cluster 2024-09-10T18:42:21.553115+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:21.554769+0000 mon.a (mon.0) 2258 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:21.555631+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:21 smithi151 bash[22565]: audit 2024-09-10T18:42:21.556041+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:22.680 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:42:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:42:23.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:22 smithi100 bash[31607]: cluster 2024-09-10T18:42:21.909038+0000 mgr.y (mgr.24512) 327 : cluster [DBG] pgmap v460: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:23.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:22 smithi100 bash[31607]: audit 2024-09-10T18:42:22.555559+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:42:23.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:22 smithi100 bash[18597]: cluster 2024-09-10T18:42:21.909038+0000 mgr.y (mgr.24512) 327 : cluster [DBG] pgmap v460: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:22 smithi100 bash[18597]: audit 2024-09-10T18:42:22.555559+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:42:23.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:22 smithi151 bash[22565]: cluster 2024-09-10T18:42:21.909038+0000 mgr.y (mgr.24512) 327 : cluster [DBG] pgmap v460: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:23.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:22 smithi151 bash[22565]: audit 2024-09-10T18:42:22.555559+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:42:24.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[31607]: audit 2024-09-10T18:42:22.304184+0000 mgr.y (mgr.24512) 328 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:24.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[31607]: audit 2024-09-10T18:42:22.539928+0000 mon.b (mon.1) 218 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:42:24.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[31607]: cluster 2024-09-10T18:42:22.980937+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[31607]: audit 2024-09-10T18:42:22.982947+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[31607]: audit 2024-09-10T18:42:23.558136+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[31607]: audit 2024-09-10T18:42:23.558310+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[31607]: cluster 2024-09-10T18:42:23.558419+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[18597]: audit 2024-09-10T18:42:22.304184+0000 mgr.y (mgr.24512) 328 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[18597]: audit 2024-09-10T18:42:22.539928+0000 mon.b (mon.1) 218 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[18597]: cluster 2024-09-10T18:42:22.980937+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[18597]: audit 2024-09-10T18:42:22.982947+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[18597]: audit 2024-09-10T18:42:23.558136+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[18597]: audit 2024-09-10T18:42:23.558310+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:42:24.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:23 smithi100 bash[18597]: cluster 2024-09-10T18:42:23.558419+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:42:24.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:23 smithi151 bash[22565]: audit 2024-09-10T18:42:22.304184+0000 mgr.y (mgr.24512) 328 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:24.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:23 smithi151 bash[22565]: audit 2024-09-10T18:42:22.539928+0000 mon.b (mon.1) 218 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:42:24.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:23 smithi151 bash[22565]: cluster 2024-09-10T18:42:22.980937+0000 mon.a (mon.0) 2261 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:42:24.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:23 smithi151 bash[22565]: audit 2024-09-10T18:42:22.982947+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:42:24.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:23 smithi151 bash[22565]: audit 2024-09-10T18:42:23.558136+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-72601-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:24.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:23 smithi151 bash[22565]: audit 2024-09-10T18:42:23.558310+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:42:24.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:23 smithi151 bash[22565]: cluster 2024-09-10T18:42:23.558419+0000 mon.a (mon.0) 2265 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:42:24.930 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:42:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:25.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[18597]: cluster 2024-09-10T18:42:23.909959+0000 mgr.y (mgr.24512) 329 : cluster [DBG] pgmap v463: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:25.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[18597]: audit 2024-09-10T18:42:24.328427+0000 mon.c (mon.2) 399 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:25.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[18597]: cluster 2024-09-10T18:42:24.550885+0000 mon.a (mon.0) 2266 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:25.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[18597]: cluster 2024-09-10T18:42:24.564160+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:42:25.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[31607]: cluster 2024-09-10T18:42:23.909959+0000 mgr.y (mgr.24512) 329 : cluster [DBG] pgmap v463: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:25.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[31607]: audit 2024-09-10T18:42:24.328427+0000 mon.c (mon.2) 399 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:25.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[31607]: cluster 2024-09-10T18:42:24.550885+0000 mon.a (mon.0) 2266 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:25.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:24 smithi100 bash[31607]: cluster 2024-09-10T18:42:24.564160+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:42:25.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:24 smithi151 bash[22565]: cluster 2024-09-10T18:42:23.909959+0000 mgr.y (mgr.24512) 329 : cluster [DBG] pgmap v463: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:25.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:24 smithi151 bash[22565]: audit 2024-09-10T18:42:24.328427+0000 mon.c (mon.2) 399 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:25.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:24 smithi151 bash[22565]: cluster 2024-09-10T18:42:24.550885+0000 mon.a (mon.0) 2266 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:25.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:24 smithi151 bash[22565]: cluster 2024-09-10T18:42:24.564160+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:42:26.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:26 smithi151 bash[22565]: cluster 2024-09-10T18:42:25.570753+0000 mon.a (mon.0) 2268 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:42:26.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:26 smithi151 bash[22565]: audit 2024-09-10T18:42:25.572765+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:26.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:26 smithi151 bash[22565]: audit 2024-09-10T18:42:25.574850+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:26.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:26 smithi151 bash[22565]: cluster 2024-09-10T18:42:25.910779+0000 mgr.y (mgr.24512) 330 : cluster [DBG] pgmap v466: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[18597]: cluster 2024-09-10T18:42:25.570753+0000 mon.a (mon.0) 2268 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:42:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[18597]: audit 2024-09-10T18:42:25.572765+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[18597]: audit 2024-09-10T18:42:25.574850+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[18597]: cluster 2024-09-10T18:42:25.910779+0000 mgr.y (mgr.24512) 330 : cluster [DBG] pgmap v466: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[31607]: cluster 2024-09-10T18:42:25.570753+0000 mon.a (mon.0) 2268 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:42:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[31607]: audit 2024-09-10T18:42:25.572765+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[31607]: audit 2024-09-10T18:42:25.574850+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:26 smithi100 bash[31607]: cluster 2024-09-10T18:42:25.910779+0000 mgr.y (mgr.24512) 330 : cluster [DBG] pgmap v466: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:27.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:27 smithi151 bash[22565]: audit 2024-09-10T18:42:26.585762+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:27.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:27 smithi151 bash[22565]: cluster 2024-09-10T18:42:26.585859+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:42:27.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:27 smithi151 bash[22565]: audit 2024-09-10T18:42:26.588034+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:27.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:27 smithi151 bash[22565]: audit 2024-09-10T18:42:26.588573+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[18597]: audit 2024-09-10T18:42:26.585762+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[18597]: cluster 2024-09-10T18:42:26.585859+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:42:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[18597]: audit 2024-09-10T18:42:26.588034+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[18597]: audit 2024-09-10T18:42:26.588573+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[31607]: audit 2024-09-10T18:42:26.585762+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[31607]: cluster 2024-09-10T18:42:26.585859+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:42:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[31607]: audit 2024-09-10T18:42:26.588034+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.100:0/4054271801' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:27 smithi100 bash[31607]: audit 2024-09-10T18:42:26.588573+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]: dispatch 2024-09-10T18:42:28.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:28 smithi151 bash[22565]: audit 2024-09-10T18:42:27.586583+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:28.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:28 smithi151 bash[22565]: cluster 2024-09-10T18:42:27.587279+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:42:28.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:28 smithi151 bash[22565]: audit 2024-09-10T18:42:27.616155+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:28.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:28 smithi151 bash[22565]: audit 2024-09-10T18:42:27.617575+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:28.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:28 smithi151 bash[22565]: audit 2024-09-10T18:42:27.618022+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:28.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:28 smithi151 bash[22565]: cluster 2024-09-10T18:42:27.911635+0000 mgr.y (mgr.24512) 331 : cluster [DBG] pgmap v469: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:29.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[18597]: audit 2024-09-10T18:42:27.586583+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:29.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[18597]: cluster 2024-09-10T18:42:27.587279+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[18597]: audit 2024-09-10T18:42:27.616155+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[18597]: audit 2024-09-10T18:42:27.617575+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[18597]: audit 2024-09-10T18:42:27.618022+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[18597]: cluster 2024-09-10T18:42:27.911635+0000 mgr.y (mgr.24512) 331 : cluster [DBG] pgmap v469: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[31607]: audit 2024-09-10T18:42:27.586583+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-72601-58"}]': finished 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[31607]: cluster 2024-09-10T18:42:27.587279+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[31607]: audit 2024-09-10T18:42:27.616155+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[31607]: audit 2024-09-10T18:42:27.617575+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[31607]: audit 2024-09-10T18:42:27.618022+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:28 smithi100 bash[31607]: cluster 2024-09-10T18:42:27.911635+0000 mgr.y (mgr.24512) 331 : cluster [DBG] pgmap v469: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:30 smithi100 bash[18597]: audit 2024-09-10T18:42:28.605460+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:30 smithi100 bash[18597]: cluster 2024-09-10T18:42:28.605574+0000 mon.a (mon.0) 2279 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:42:30.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:30 smithi100 bash[18597]: audit 2024-09-10T18:42:28.609545+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-72601-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:30 smithi100 bash[31607]: audit 2024-09-10T18:42:28.605460+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:30 smithi100 bash[31607]: cluster 2024-09-10T18:42:28.605574+0000 mon.a (mon.0) 2279 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:42:30.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:30 smithi100 bash[31607]: audit 2024-09-10T18:42:28.609545+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-72601-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:30.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:30 smithi151 bash[22565]: audit 2024-09-10T18:42:28.605460+0000 mon.a (mon.0) 2278 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:30.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:30 smithi151 bash[22565]: cluster 2024-09-10T18:42:28.605574+0000 mon.a (mon.0) 2279 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:42:30.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:30 smithi151 bash[22565]: audit 2024-09-10T18:42:28.609545+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-72601-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[18597]: cluster 2024-09-10T18:42:29.912427+0000 mgr.y (mgr.24512) 332 : cluster [DBG] pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[18597]: cluster 2024-09-10T18:42:30.033278+0000 mon.a (mon.0) 2281 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[18597]: cluster 2024-09-10T18:42:30.360257+0000 mon.a (mon.0) 2282 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[18597]: audit 2024-09-10T18:42:30.621953+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-72601-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[18597]: cluster 2024-09-10T18:42:30.622044+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[31607]: cluster 2024-09-10T18:42:29.912427+0000 mgr.y (mgr.24512) 332 : cluster [DBG] pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[31607]: cluster 2024-09-10T18:42:30.033278+0000 mon.a (mon.0) 2281 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[31607]: cluster 2024-09-10T18:42:30.360257+0000 mon.a (mon.0) 2282 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[31607]: audit 2024-09-10T18:42:30.621953+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-72601-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[31607]: cluster 2024-09-10T18:42:30.622044+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:42:31.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:31 smithi151 bash[22565]: cluster 2024-09-10T18:42:29.912427+0000 mgr.y (mgr.24512) 332 : cluster [DBG] pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:31.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:31 smithi151 bash[22565]: cluster 2024-09-10T18:42:30.033278+0000 mon.a (mon.0) 2281 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:42:31.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:31 smithi151 bash[22565]: cluster 2024-09-10T18:42:30.360257+0000 mon.a (mon.0) 2282 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:31.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:31 smithi151 bash[22565]: audit 2024-09-10T18:42:30.621953+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-72601-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:31.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:31 smithi151 bash[22565]: cluster 2024-09-10T18:42:30.622044+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:42:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:42:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:31] "GET /metrics HTTP/1.1" 200 230910 "" "Prometheus/2.43.0" 2024-09-10T18:42:32.630 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:42:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:42:32.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:32 smithi151 bash[22565]: cluster 2024-09-10T18:42:31.630768+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:42:32.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:32 smithi151 bash[22565]: cluster 2024-09-10T18:42:31.913104+0000 mgr.y (mgr.24512) 333 : cluster [DBG] pgmap v475: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:42:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:32 smithi100 bash[18597]: cluster 2024-09-10T18:42:31.630768+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:42:33.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:32 smithi100 bash[18597]: cluster 2024-09-10T18:42:31.913104+0000 mgr.y (mgr.24512) 333 : cluster [DBG] pgmap v475: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:42:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:32 smithi100 bash[31607]: cluster 2024-09-10T18:42:31.630768+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:42:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:32 smithi100 bash[31607]: cluster 2024-09-10T18:42:31.913104+0000 mgr.y (mgr.24512) 333 : cluster [DBG] pgmap v475: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:42:33.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:33 smithi151 bash[22565]: audit 2024-09-10T18:42:32.315302+0000 mgr.y (mgr.24512) 334 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:33.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:33 smithi151 bash[22565]: cluster 2024-09-10T18:42:32.635722+0000 mon.a (mon.0) 2286 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:42:33.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:33 smithi151 bash[22565]: audit 2024-09-10T18:42:32.638419+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:34.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:33 smithi100 bash[18597]: audit 2024-09-10T18:42:32.315302+0000 mgr.y (mgr.24512) 334 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:34.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:33 smithi100 bash[18597]: cluster 2024-09-10T18:42:32.635722+0000 mon.a (mon.0) 2286 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:42:34.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:33 smithi100 bash[18597]: audit 2024-09-10T18:42:32.638419+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:34.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:33 smithi100 bash[31607]: audit 2024-09-10T18:42:32.315302+0000 mgr.y (mgr.24512) 334 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:34.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:33 smithi100 bash[31607]: cluster 2024-09-10T18:42:32.635722+0000 mon.a (mon.0) 2286 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:42:34.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:33 smithi100 bash[31607]: audit 2024-09-10T18:42:32.638419+0000 mon.a (mon.0) 2287 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:34.929 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:42:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:34.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:34 smithi151 bash[22565]: audit 2024-09-10T18:42:33.638619+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:34.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:34 smithi151 bash[22565]: cluster 2024-09-10T18:42:33.638712+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:42:34.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:34 smithi151 bash[22565]: audit 2024-09-10T18:42:33.640302+0000 mon.a (mon.0) 2290 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:34.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:34 smithi151 bash[22565]: cluster 2024-09-10T18:42:33.913965+0000 mgr.y (mgr.24512) 335 : cluster [DBG] pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:42:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[18597]: audit 2024-09-10T18:42:33.638619+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[18597]: cluster 2024-09-10T18:42:33.638712+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:42:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[18597]: audit 2024-09-10T18:42:33.640302+0000 mon.a (mon.0) 2290 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:35.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[18597]: cluster 2024-09-10T18:42:33.913965+0000 mgr.y (mgr.24512) 335 : cluster [DBG] pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:42:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[31607]: audit 2024-09-10T18:42:33.638619+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[31607]: cluster 2024-09-10T18:42:33.638712+0000 mon.a (mon.0) 2289 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:42:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[31607]: audit 2024-09-10T18:42:33.640302+0000 mon.a (mon.0) 2290 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]: dispatch 2024-09-10T18:42:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:34 smithi100 bash[31607]: cluster 2024-09-10T18:42:33.913965+0000 mgr.y (mgr.24512) 335 : cluster [DBG] pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:42:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:35 smithi151 bash[22565]: audit 2024-09-10T18:42:34.646231+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:35.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:35 smithi151 bash[22565]: cluster 2024-09-10T18:42:34.646371+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:42:36.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:35 smithi100 bash[31607]: audit 2024-09-10T18:42:34.646231+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:36.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:35 smithi100 bash[31607]: cluster 2024-09-10T18:42:34.646371+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:42:36.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:35 smithi100 bash[18597]: audit 2024-09-10T18:42:34.646231+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? 172.21.15.100:0/892265809' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-72601-59"}]': finished 2024-09-10T18:42:36.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:35 smithi100 bash[18597]: cluster 2024-09-10T18:42:34.646371+0000 mon.a (mon.0) 2292 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:42:36.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: audit 2024-09-10T18:42:35.635097+0000 mon.b (mon.1) 219 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:36.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: audit 2024-09-10T18:42:35.654185+0000 mon.b (mon.1) 220 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]: dispatch 2024-09-10T18:42:36.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: audit 2024-09-10T18:42:35.662086+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:36.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: cluster 2024-09-10T18:42:35.669910+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:42:36.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: audit 2024-09-10T18:42:35.673971+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]: dispatch 2024-09-10T18:42:36.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: audit 2024-09-10T18:42:35.680836+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.100:0/631275421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:36.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: audit 2024-09-10T18:42:35.681556+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:36.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:36 smithi151 bash[22565]: cluster 2024-09-10T18:42:35.914947+0000 mgr.y (mgr.24512) 336 : cluster [DBG] pgmap v481: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:42:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: audit 2024-09-10T18:42:35.635097+0000 mon.b (mon.1) 219 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: audit 2024-09-10T18:42:35.654185+0000 mon.b (mon.1) 220 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]: dispatch 2024-09-10T18:42:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: audit 2024-09-10T18:42:35.662086+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: cluster 2024-09-10T18:42:35.669910+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:42:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: audit 2024-09-10T18:42:35.673971+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]: dispatch 2024-09-10T18:42:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: audit 2024-09-10T18:42:35.680836+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.100:0/631275421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: audit 2024-09-10T18:42:35.681556+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[18597]: cluster 2024-09-10T18:42:35.914947+0000 mgr.y (mgr.24512) 336 : cluster [DBG] pgmap v481: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: audit 2024-09-10T18:42:35.635097+0000 mon.b (mon.1) 219 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: audit 2024-09-10T18:42:35.654185+0000 mon.b (mon.1) 220 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: audit 2024-09-10T18:42:35.662086+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: cluster 2024-09-10T18:42:35.669910+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: audit 2024-09-10T18:42:35.673971+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: audit 2024-09-10T18:42:35.680836+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.100:0/631275421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: audit 2024-09-10T18:42:35.681556+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:36 smithi100 bash[31607]: cluster 2024-09-10T18:42:35.914947+0000 mgr.y (mgr.24512) 336 : cluster [DBG] pgmap v481: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: cluster 2024-09-10T18:42:36.664840+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: audit 2024-09-10T18:42:36.676446+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]': finished 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: audit 2024-09-10T18:42:36.676658+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: cluster 2024-09-10T18:42:36.676843+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: audit 2024-09-10T18:42:36.918494+0000 mon.c (mon.2) 403 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: audit 2024-09-10T18:42:37.371740+0000 mon.c (mon.2) 404 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: audit 2024-09-10T18:42:37.373360+0000 mon.c (mon.2) 405 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:37.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:37 smithi151 bash[22565]: audit 2024-09-10T18:42:37.380868+0000 mon.a (mon.0) 2301 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:42:38.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: cluster 2024-09-10T18:42:36.664840+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: audit 2024-09-10T18:42:36.676446+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]': finished 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: audit 2024-09-10T18:42:36.676658+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: cluster 2024-09-10T18:42:36.676843+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: audit 2024-09-10T18:42:36.918494+0000 mon.c (mon.2) 403 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: audit 2024-09-10T18:42:37.371740+0000 mon.c (mon.2) 404 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: audit 2024-09-10T18:42:37.373360+0000 mon.c (mon.2) 405 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[18597]: audit 2024-09-10T18:42:37.380868+0000 mon.a (mon.0) 2301 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: cluster 2024-09-10T18:42:36.664840+0000 mon.a (mon.0) 2297 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: audit 2024-09-10T18:42:36.676446+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-45"}]': finished 2024-09-10T18:42:38.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: audit 2024-09-10T18:42:36.676658+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-72601-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:38.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: cluster 2024-09-10T18:42:36.676843+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:42:38.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: audit 2024-09-10T18:42:36.918494+0000 mon.c (mon.2) 403 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:38.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: audit 2024-09-10T18:42:37.371740+0000 mon.c (mon.2) 404 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:38.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: audit 2024-09-10T18:42:37.373360+0000 mon.c (mon.2) 405 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:38.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:37 smithi100 bash[31607]: audit 2024-09-10T18:42:37.380868+0000 mon.a (mon.0) 2301 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:42:39.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: cluster 2024-09-10T18:42:37.678042+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: audit 2024-09-10T18:42:37.709876+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: cluster 2024-09-10T18:42:37.915726+0000 mgr.y (mgr.24512) 337 : cluster [DBG] pgmap v484: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: audit 2024-09-10T18:42:38.107178+0000 mon.a (mon.0) 2303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: audit 2024-09-10T18:42:38.108291+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: audit 2024-09-10T18:42:38.108880+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: audit 2024-09-10T18:42:38.109825+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[18597]: audit 2024-09-10T18:42:38.110435+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: cluster 2024-09-10T18:42:37.678042+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:42:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: audit 2024-09-10T18:42:37.709876+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: cluster 2024-09-10T18:42:37.915726+0000 mgr.y (mgr.24512) 337 : cluster [DBG] pgmap v484: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: audit 2024-09-10T18:42:38.107178+0000 mon.a (mon.0) 2303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: audit 2024-09-10T18:42:38.108291+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: audit 2024-09-10T18:42:38.108880+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: audit 2024-09-10T18:42:38.109825+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:38 smithi100 bash[31607]: audit 2024-09-10T18:42:38.110435+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:39.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: cluster 2024-09-10T18:42:37.678042+0000 mon.a (mon.0) 2302 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:42:39.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: audit 2024-09-10T18:42:37.709876+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: cluster 2024-09-10T18:42:37.915726+0000 mgr.y (mgr.24512) 337 : cluster [DBG] pgmap v484: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:39.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: audit 2024-09-10T18:42:38.107178+0000 mon.a (mon.0) 2303 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: audit 2024-09-10T18:42:38.108291+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: audit 2024-09-10T18:42:38.108880+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:39.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: audit 2024-09-10T18:42:38.109825+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:39.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:38 smithi151 bash[22565]: audit 2024-09-10T18:42:38.110435+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:40.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:40 smithi151 bash[22565]: audit 2024-09-10T18:42:38.684239+0000 mon.b (mon.1) 221 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:40 smithi151 bash[22565]: audit 2024-09-10T18:42:38.699130+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:40.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:40 smithi151 bash[22565]: cluster 2024-09-10T18:42:38.699492+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:42:40.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:40 smithi151 bash[22565]: audit 2024-09-10T18:42:38.701857+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:40.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:40 smithi151 bash[22565]: audit 2024-09-10T18:42:38.707187+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:40.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:40 smithi151 bash[22565]: audit 2024-09-10T18:42:38.708075+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:40 smithi151 bash[22565]: audit 2024-09-10T18:42:39.329007+0000 mon.c (mon.2) 410 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:40.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[31607]: audit 2024-09-10T18:42:38.684239+0000 mon.b (mon.1) 221 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[31607]: audit 2024-09-10T18:42:38.699130+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:40.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[31607]: cluster 2024-09-10T18:42:38.699492+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[31607]: audit 2024-09-10T18:42:38.701857+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[31607]: audit 2024-09-10T18:42:38.707187+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[31607]: audit 2024-09-10T18:42:38.708075+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[31607]: audit 2024-09-10T18:42:39.329007+0000 mon.c (mon.2) 410 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[18597]: audit 2024-09-10T18:42:38.684239+0000 mon.b (mon.1) 221 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[18597]: audit 2024-09-10T18:42:38.699130+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-72601-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[18597]: cluster 2024-09-10T18:42:38.699492+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[18597]: audit 2024-09-10T18:42:38.701857+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[18597]: audit 2024-09-10T18:42:38.707187+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[18597]: audit 2024-09-10T18:42:38.708075+0000 mon.a (mon.0) 2309 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:40 smithi100 bash[18597]: audit 2024-09-10T18:42:39.329007+0000 mon.c (mon.2) 410 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:41.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: cluster 2024-09-10T18:42:39.916479+0000 mgr.y (mgr.24512) 338 : cluster [DBG] pgmap v487: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: audit 2024-09-10T18:42:40.125785+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: cluster 2024-09-10T18:42:40.125919+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: audit 2024-09-10T18:42:40.163329+0000 mon.b (mon.1) 222 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: audit 2024-09-10T18:42:40.181366+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: audit 2024-09-10T18:42:40.693894+0000 mon.b (mon.1) 223 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: audit 2024-09-10T18:42:40.709652+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: audit 2024-09-10T18:42:40.710085+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: cluster 2024-09-10T18:42:40.710246+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:42:41.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:41 smithi151 bash[22565]: audit 2024-09-10T18:42:40.713098+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:41.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:41] "GET /metrics HTTP/1.1" 200 228520 "" "Prometheus/2.43.0" 2024-09-10T18:42:41.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: cluster 2024-09-10T18:42:39.916479+0000 mgr.y (mgr.24512) 338 : cluster [DBG] pgmap v487: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: audit 2024-09-10T18:42:40.125785+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: cluster 2024-09-10T18:42:40.125919+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: audit 2024-09-10T18:42:40.163329+0000 mon.b (mon.1) 222 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: audit 2024-09-10T18:42:40.181366+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: audit 2024-09-10T18:42:40.693894+0000 mon.b (mon.1) 223 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: audit 2024-09-10T18:42:40.709652+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: audit 2024-09-10T18:42:40.710085+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: cluster 2024-09-10T18:42:40.710246+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[31607]: audit 2024-09-10T18:42:40.713098+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: cluster 2024-09-10T18:42:39.916479+0000 mgr.y (mgr.24512) 338 : cluster [DBG] pgmap v487: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: audit 2024-09-10T18:42:40.125785+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: cluster 2024-09-10T18:42:40.125919+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: audit 2024-09-10T18:42:40.163329+0000 mon.b (mon.1) 222 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: audit 2024-09-10T18:42:40.181366+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: audit 2024-09-10T18:42:40.693894+0000 mon.b (mon.1) 223 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: audit 2024-09-10T18:42:40.709652+0000 mon.a (mon.0) 2313 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-72601-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: audit 2024-09-10T18:42:40.710085+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: cluster 2024-09-10T18:42:40.710246+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:42:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:41 smithi100 bash[18597]: audit 2024-09-10T18:42:40.713098+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:42.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:42:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:42:43.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[18597]: audit 2024-09-10T18:42:41.696650+0000 mon.b (mon.1) 224 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]: dispatch 2024-09-10T18:42:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[18597]: audit 2024-09-10T18:42:41.711772+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]': finished 2024-09-10T18:42:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[18597]: cluster 2024-09-10T18:42:41.712271+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:42:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[18597]: audit 2024-09-10T18:42:41.715012+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]: dispatch 2024-09-10T18:42:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[18597]: cluster 2024-09-10T18:42:41.917346+0000 mgr.y (mgr.24512) 339 : cluster [DBG] pgmap v490: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:42:43.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[31607]: audit 2024-09-10T18:42:41.696650+0000 mon.b (mon.1) 224 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]: dispatch 2024-09-10T18:42:43.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[31607]: audit 2024-09-10T18:42:41.711772+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]': finished 2024-09-10T18:42:43.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[31607]: cluster 2024-09-10T18:42:41.712271+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:42:43.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[31607]: audit 2024-09-10T18:42:41.715012+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]: dispatch 2024-09-10T18:42:43.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:42 smithi100 bash[31607]: cluster 2024-09-10T18:42:41.917346+0000 mgr.y (mgr.24512) 339 : cluster [DBG] pgmap v490: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:42:43.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:42 smithi151 bash[22565]: audit 2024-09-10T18:42:41.696650+0000 mon.b (mon.1) 224 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]: dispatch 2024-09-10T18:42:43.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:42 smithi151 bash[22565]: audit 2024-09-10T18:42:41.711772+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-47"}]': finished 2024-09-10T18:42:43.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:42 smithi151 bash[22565]: cluster 2024-09-10T18:42:41.712271+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:42:43.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:42 smithi151 bash[22565]: audit 2024-09-10T18:42:41.715012+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]: dispatch 2024-09-10T18:42:43.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:42 smithi151 bash[22565]: cluster 2024-09-10T18:42:41.917346+0000 mgr.y (mgr.24512) 339 : cluster [DBG] pgmap v490: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:42:44.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: audit 2024-09-10T18:42:42.326438+0000 mgr.y (mgr.24512) 340 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: audit 2024-09-10T18:42:42.697620+0000 mon.b (mon.1) 225 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: cluster 2024-09-10T18:42:42.703645+0000 mon.a (mon.0) 2320 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: cluster 2024-09-10T18:42:42.704352+0000 mon.a (mon.0) 2321 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: audit 2024-09-10T18:42:42.712719+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]': finished 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: cluster 2024-09-10T18:42:42.712891+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: audit 2024-09-10T18:42:42.715773+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: audit 2024-09-10T18:42:42.722453+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[18597]: audit 2024-09-10T18:42:42.723250+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: audit 2024-09-10T18:42:42.326438+0000 mgr.y (mgr.24512) 340 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: audit 2024-09-10T18:42:42.697620+0000 mon.b (mon.1) 225 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: cluster 2024-09-10T18:42:42.703645+0000 mon.a (mon.0) 2320 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: cluster 2024-09-10T18:42:42.704352+0000 mon.a (mon.0) 2321 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: audit 2024-09-10T18:42:42.712719+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]': finished 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: cluster 2024-09-10T18:42:42.712891+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: audit 2024-09-10T18:42:42.715773+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: audit 2024-09-10T18:42:42.722453+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:44.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:43 smithi100 bash[31607]: audit 2024-09-10T18:42:42.723250+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: audit 2024-09-10T18:42:42.326438+0000 mgr.y (mgr.24512) 340 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:44.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: audit 2024-09-10T18:42:42.697620+0000 mon.b (mon.1) 225 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:44.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: cluster 2024-09-10T18:42:42.703645+0000 mon.a (mon.0) 2320 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:44.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: cluster 2024-09-10T18:42:42.704352+0000 mon.a (mon.0) 2321 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:44.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: audit 2024-09-10T18:42:42.712719+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-47", "mode": "writeback"}]': finished 2024-09-10T18:42:44.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: cluster 2024-09-10T18:42:42.712891+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:42:44.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: audit 2024-09-10T18:42:42.715773+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: audit 2024-09-10T18:42:42.722453+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:42:44.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:43 smithi151 bash[22565]: audit 2024-09-10T18:42:42.723250+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.929 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:44.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: audit 2024-09-10T18:42:43.723263+0000 mon.b (mon.1) 226 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:44.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: audit 2024-09-10T18:42:43.734186+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:42:44.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: audit 2024-09-10T18:42:43.734330+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:44.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: cluster 2024-09-10T18:42:43.734444+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:42:44.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: audit 2024-09-10T18:42:43.738062+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: audit 2024-09-10T18:42:43.738454+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:44.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: audit 2024-09-10T18:42:43.741042+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:44.930 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:44 smithi151 bash[22565]: cluster 2024-09-10T18:42:43.918238+0000 mgr.y (mgr.24512) 341 : cluster [DBG] pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:42:45.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: audit 2024-09-10T18:42:43.723263+0000 mon.b (mon.1) 226 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: audit 2024-09-10T18:42:43.734186+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: audit 2024-09-10T18:42:43.734330+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: cluster 2024-09-10T18:42:43.734444+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: audit 2024-09-10T18:42:43.738062+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: audit 2024-09-10T18:42:43.738454+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: audit 2024-09-10T18:42:43.741042+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[31607]: cluster 2024-09-10T18:42:43.918238+0000 mgr.y (mgr.24512) 341 : cluster [DBG] pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: audit 2024-09-10T18:42:43.723263+0000 mon.b (mon.1) 226 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: audit 2024-09-10T18:42:43.734186+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:42:45.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: audit 2024-09-10T18:42:43.734330+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:45.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: cluster 2024-09-10T18:42:43.734444+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:42:45.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: audit 2024-09-10T18:42:43.738062+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.100:0/1470195780' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:45.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: audit 2024-09-10T18:42:43.738454+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]: dispatch 2024-09-10T18:42:45.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: audit 2024-09-10T18:42:43.741042+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:42:45.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:44 smithi100 bash[18597]: cluster 2024-09-10T18:42:43.918238+0000 mgr.y (mgr.24512) 341 : cluster [DBG] pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:42:46.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.738008+0000 mon.b (mon.1) 227 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.753702+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.753814+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: cluster 2024-09-10T18:42:44.753888+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.755930+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.776047+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.776593+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.777331+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.777763+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.778451+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:46.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[18597]: audit 2024-09-10T18:42:44.778844+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.738008+0000 mon.b (mon.1) 227 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.753702+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.753814+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: cluster 2024-09-10T18:42:44.753888+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.755930+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.776047+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.776593+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.777331+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.777763+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.778451+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:46.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:45 smithi100 bash[31607]: audit 2024-09-10T18:42:44.778844+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:46.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.738008+0000 mon.b (mon.1) 227 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:46.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.753702+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-72601-61"}]': finished 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.753814+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: cluster 2024-09-10T18:42:44.753888+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.755930+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.776047+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.776593+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.777331+0000 mon.c (mon.2) 414 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.777763+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.778451+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:46.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:45 smithi151 bash[22565]: audit 2024-09-10T18:42:44.778844+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:47.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: audit 2024-09-10T18:42:45.735784+0000 mon.b (mon.1) 228 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: cluster 2024-09-10T18:42:45.741328+0000 mon.a (mon.0) 2338 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: audit 2024-09-10T18:42:45.751491+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: audit 2024-09-10T18:42:45.751735+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: cluster 2024-09-10T18:42:45.751950+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: audit 2024-09-10T18:42:45.752964+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: audit 2024-09-10T18:42:45.753595+0000 mon.a (mon.0) 2342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: audit 2024-09-10T18:42:45.755218+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[18597]: cluster 2024-09-10T18:42:45.919073+0000 mgr.y (mgr.24512) 342 : cluster [DBG] pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: audit 2024-09-10T18:42:45.735784+0000 mon.b (mon.1) 228 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: cluster 2024-09-10T18:42:45.741328+0000 mon.a (mon.0) 2338 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: audit 2024-09-10T18:42:45.751491+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: audit 2024-09-10T18:42:45.751735+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: cluster 2024-09-10T18:42:45.751950+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: audit 2024-09-10T18:42:45.752964+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: audit 2024-09-10T18:42:45.753595+0000 mon.a (mon.0) 2342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: audit 2024-09-10T18:42:45.755218+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:42:47.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:47 smithi100 bash[31607]: cluster 2024-09-10T18:42:45.919073+0000 mgr.y (mgr.24512) 342 : cluster [DBG] pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:47.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: audit 2024-09-10T18:42:45.735784+0000 mon.b (mon.1) 228 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: cluster 2024-09-10T18:42:45.741328+0000 mon.a (mon.0) 2338 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: audit 2024-09-10T18:42:45.751491+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: audit 2024-09-10T18:42:45.751735+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: cluster 2024-09-10T18:42:45.751950+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: audit 2024-09-10T18:42:45.752964+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: audit 2024-09-10T18:42:45.753595+0000 mon.a (mon.0) 2342 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: audit 2024-09-10T18:42:45.755218+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:42:47.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:47 smithi151 bash[22565]: cluster 2024-09-10T18:42:45.919073+0000 mgr.y (mgr.24512) 342 : cluster [DBG] pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: audit 2024-09-10T18:42:47.161863+0000 mon.b (mon.1) 229 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: audit 2024-09-10T18:42:47.177330+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: cluster 2024-09-10T18:42:47.177602+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: audit 2024-09-10T18:42:47.190329+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: audit 2024-09-10T18:42:47.751187+0000 mon.b (mon.1) 230 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: audit 2024-09-10T18:42:47.765860+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: audit 2024-09-10T18:42:47.766217+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: cluster 2024-09-10T18:42:47.766442+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: audit 2024-09-10T18:42:47.769623+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[18597]: cluster 2024-09-10T18:42:47.919959+0000 mgr.y (mgr.24512) 343 : cluster [DBG] pgmap v499: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: audit 2024-09-10T18:42:47.161863+0000 mon.b (mon.1) 229 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:42:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: audit 2024-09-10T18:42:47.177330+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: cluster 2024-09-10T18:42:47.177602+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: audit 2024-09-10T18:42:47.190329+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: audit 2024-09-10T18:42:47.751187+0000 mon.b (mon.1) 230 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: audit 2024-09-10T18:42:47.765860+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: audit 2024-09-10T18:42:47.766217+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: cluster 2024-09-10T18:42:47.766442+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: audit 2024-09-10T18:42:47.769623+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:42:48.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:48 smithi100 bash[31607]: cluster 2024-09-10T18:42:47.919959+0000 mgr.y (mgr.24512) 343 : cluster [DBG] pgmap v499: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: audit 2024-09-10T18:42:47.161863+0000 mon.b (mon.1) 229 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:42:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: audit 2024-09-10T18:42:47.177330+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:42:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: cluster 2024-09-10T18:42:47.177602+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:42:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: audit 2024-09-10T18:42:47.190329+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:42:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: audit 2024-09-10T18:42:47.751187+0000 mon.b (mon.1) 230 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:42:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: audit 2024-09-10T18:42:47.765860+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-72601-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: audit 2024-09-10T18:42:47.766217+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:42:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: cluster 2024-09-10T18:42:47.766442+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:42:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: audit 2024-09-10T18:42:47.769623+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:42:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:48 smithi151 bash[22565]: cluster 2024-09-10T18:42:47.919959+0000 mgr.y (mgr.24512) 343 : cluster [DBG] pgmap v499: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:49.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[18597]: cluster 2024-09-10T18:42:48.757835+0000 mon.a (mon.0) 2351 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:49.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[18597]: audit 2024-09-10T18:42:48.766531+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[18597]: cluster 2024-09-10T18:42:48.766989+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[18597]: audit 2024-09-10T18:42:48.793063+0000 mon.b (mon.1) 231 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[18597]: audit 2024-09-10T18:42:48.810738+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[31607]: cluster 2024-09-10T18:42:48.757835+0000 mon.a (mon.0) 2351 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[31607]: audit 2024-09-10T18:42:48.766531+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[31607]: cluster 2024-09-10T18:42:48.766989+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[31607]: audit 2024-09-10T18:42:48.793063+0000 mon.b (mon.1) 231 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:49 smithi100 bash[31607]: audit 2024-09-10T18:42:48.810738+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:49 smithi151 bash[22565]: cluster 2024-09-10T18:42:48.757835+0000 mon.a (mon.0) 2351 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:49 smithi151 bash[22565]: audit 2024-09-10T18:42:48.766531+0000 mon.a (mon.0) 2352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:42:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:49 smithi151 bash[22565]: cluster 2024-09-10T18:42:48.766989+0000 mon.a (mon.0) 2353 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:42:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:49 smithi151 bash[22565]: audit 2024-09-10T18:42:48.793063+0000 mon.b (mon.1) 231 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:49.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:49 smithi151 bash[22565]: audit 2024-09-10T18:42:48.810738+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:51.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[18597]: audit 2024-09-10T18:42:49.767436+0000 mon.b (mon.1) 232 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[18597]: audit 2024-09-10T18:42:49.781939+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[18597]: cluster 2024-09-10T18:42:49.782181+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[18597]: audit 2024-09-10T18:42:49.785053+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[18597]: audit 2024-09-10T18:42:49.785968+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[18597]: audit 2024-09-10T18:42:49.786335+0000 mon.a (mon.0) 2358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[18597]: cluster 2024-09-10T18:42:49.920808+0000 mgr.y (mgr.24512) 344 : cluster [DBG] pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[31607]: audit 2024-09-10T18:42:49.767436+0000 mon.b (mon.1) 232 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[31607]: audit 2024-09-10T18:42:49.781939+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:42:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[31607]: cluster 2024-09-10T18:42:49.782181+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:42:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[31607]: audit 2024-09-10T18:42:49.785053+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[31607]: audit 2024-09-10T18:42:49.785968+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[31607]: audit 2024-09-10T18:42:49.786335+0000 mon.a (mon.0) 2358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:51.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:50 smithi100 bash[31607]: cluster 2024-09-10T18:42:49.920808+0000 mgr.y (mgr.24512) 344 : cluster [DBG] pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:51.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:50 smithi151 bash[22565]: audit 2024-09-10T18:42:49.767436+0000 mon.b (mon.1) 232 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:51.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:50 smithi151 bash[22565]: audit 2024-09-10T18:42:49.781939+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:42:51.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:50 smithi151 bash[22565]: cluster 2024-09-10T18:42:49.782181+0000 mon.a (mon.0) 2356 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:42:51.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:50 smithi151 bash[22565]: audit 2024-09-10T18:42:49.785053+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:51.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:50 smithi151 bash[22565]: audit 2024-09-10T18:42:49.785968+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:51.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:50 smithi151 bash[22565]: audit 2024-09-10T18:42:49.786335+0000 mon.a (mon.0) 2358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:51.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:50 smithi151 bash[22565]: cluster 2024-09-10T18:42:49.920808+0000 mgr.y (mgr.24512) 344 : cluster [DBG] pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:51] "GET /metrics HTTP/1.1" 200 228520 "" "Prometheus/2.43.0" 2024-09-10T18:42:52.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.793748+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]': finished 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.793948+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: cluster 2024-09-10T18:42:50.794134+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.796570+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.796964+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.811283+0000 mon.b (mon.1) 233 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.813177+0000 mon.b (mon.1) 234 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.829045+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:52.180 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:51 smithi151 bash[22565]: audit 2024-09-10T18:42:50.830782+0000 mon.a (mon.0) 2364 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:52.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.793748+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]': finished 2024-09-10T18:42:52.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.793948+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:52.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: cluster 2024-09-10T18:42:50.794134+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.796570+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.796964+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.811283+0000 mon.b (mon.1) 233 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.813177+0000 mon.b (mon.1) 234 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.829045+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[18597]: audit 2024-09-10T18:42:50.830782+0000 mon.a (mon.0) 2364 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.793748+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]': finished 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.793948+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: cluster 2024-09-10T18:42:50.794134+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.796570+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.100:0/383635768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.796964+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]: dispatch 2024-09-10T18:42:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.811283+0000 mon.b (mon.1) 233 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.813177+0000 mon.b (mon.1) 234 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.829045+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:42:52.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:51 smithi100 bash[31607]: audit 2024-09-10T18:42:50.830782+0000 mon.a (mon.0) 2364 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-47"}]: dispatch 2024-09-10T18:42:52.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:42:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:42:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:52 smithi151 bash[22565]: audit 2024-09-10T18:42:51.795119+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:52 smithi151 bash[22565]: cluster 2024-09-10T18:42:51.795365+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:42:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:52 smithi151 bash[22565]: cluster 2024-09-10T18:42:51.921580+0000 mgr.y (mgr.24512) 345 : cluster [DBG] pgmap v505: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:42:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:52 smithi151 bash[22565]: audit 2024-09-10T18:42:52.237188+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:52 smithi151 bash[22565]: audit 2024-09-10T18:42:52.240904+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:53.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:52 smithi151 bash[22565]: audit 2024-09-10T18:42:52.241480+0000 mon.a (mon.0) 2369 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-72601-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[18597]: audit 2024-09-10T18:42:51.795119+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[18597]: cluster 2024-09-10T18:42:51.795365+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:42:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[18597]: cluster 2024-09-10T18:42:51.921580+0000 mgr.y (mgr.24512) 345 : cluster [DBG] pgmap v505: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[18597]: audit 2024-09-10T18:42:52.237188+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[18597]: audit 2024-09-10T18:42:52.240904+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[18597]: audit 2024-09-10T18:42:52.241480+0000 mon.a (mon.0) 2369 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-72601-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[31607]: audit 2024-09-10T18:42:51.795119+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-72601-62"}]': finished 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[31607]: cluster 2024-09-10T18:42:51.795365+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[31607]: cluster 2024-09-10T18:42:51.921580+0000 mgr.y (mgr.24512) 345 : cluster [DBG] pgmap v505: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[31607]: audit 2024-09-10T18:42:52.237188+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[31607]: audit 2024-09-10T18:42:52.240904+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:52 smithi100 bash[31607]: audit 2024-09-10T18:42:52.241480+0000 mon.a (mon.0) 2369 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-72601-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[18597]: audit 2024-09-10T18:42:52.335864+0000 mgr.y (mgr.24512) 346 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[18597]: audit 2024-09-10T18:42:52.810785+0000 mon.b (mon.1) 235 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[18597]: audit 2024-09-10T18:42:52.822427+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-72601-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[18597]: cluster 2024-09-10T18:42:52.822677+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:42:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[18597]: audit 2024-09-10T18:42:52.824495+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-72601-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[18597]: audit 2024-09-10T18:42:52.829393+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[31607]: audit 2024-09-10T18:42:52.335864+0000 mgr.y (mgr.24512) 346 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:54.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[31607]: audit 2024-09-10T18:42:52.810785+0000 mon.b (mon.1) 235 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[31607]: audit 2024-09-10T18:42:52.822427+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-72601-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:54.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[31607]: cluster 2024-09-10T18:42:52.822677+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:42:54.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[31607]: audit 2024-09-10T18:42:52.824495+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-72601-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:54.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:54 smithi100 bash[31607]: audit 2024-09-10T18:42:52.829393+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.554 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:54 smithi151 bash[22565]: audit 2024-09-10T18:42:52.335864+0000 mgr.y (mgr.24512) 346 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:54.554 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:54 smithi151 bash[22565]: audit 2024-09-10T18:42:52.810785+0000 mon.b (mon.1) 235 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.554 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:54 smithi151 bash[22565]: audit 2024-09-10T18:42:52.822427+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-72601-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:54.554 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:54 smithi151 bash[22565]: cluster 2024-09-10T18:42:52.822677+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:42:54.554 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:54 smithi151 bash[22565]: audit 2024-09-10T18:42:52.824495+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-72601-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:54.554 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:54 smithi151 bash[22565]: audit 2024-09-10T18:42:52.829393+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.929 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:42:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:42:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:55.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: cluster 2024-09-10T18:42:53.922145+0000 mgr.y (mgr.24512) 347 : cluster [DBG] pgmap v508: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.247712+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.249860+0000 mon.a (mon.0) 2374 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: cluster 2024-09-10T18:42:54.250253+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.264922+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.329098+0000 mon.c (mon.2) 419 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.810867+0000 mon.b (mon.1) 237 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.824631+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-72601-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.824974+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: cluster 2024-09-10T18:42:54.825199+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:42:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[18597]: audit 2024-09-10T18:42:54.829364+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: cluster 2024-09-10T18:42:53.922145+0000 mgr.y (mgr.24512) 347 : cluster [DBG] pgmap v508: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.247712+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.249860+0000 mon.a (mon.0) 2374 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: cluster 2024-09-10T18:42:54.250253+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.264922+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.329098+0000 mon.c (mon.2) 419 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.810867+0000 mon.b (mon.1) 237 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.824631+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-72601-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.824974+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: cluster 2024-09-10T18:42:54.825199+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:42:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:55 smithi100 bash[31607]: audit 2024-09-10T18:42:54.829364+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:42:55.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: cluster 2024-09-10T18:42:53.922145+0000 mgr.y (mgr.24512) 347 : cluster [DBG] pgmap v508: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:42:55.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.247712+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:55.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.249860+0000 mon.a (mon.0) 2374 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:55.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: cluster 2024-09-10T18:42:54.250253+0000 mon.a (mon.0) 2375 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:42:55.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.264922+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:55.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.329098+0000 mon.c (mon.2) 419 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:55.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.810867+0000 mon.b (mon.1) 237 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:42:55.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.824631+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-72601-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:42:55.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.824974+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:55.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: cluster 2024-09-10T18:42:54.825199+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:42:55.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:55 smithi151 bash[22565]: audit 2024-09-10T18:42:54.829364+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:42:56.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[31607]: cluster 2024-09-10T18:42:55.367564+0000 mon.a (mon.0) 2381 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[31607]: audit 2024-09-10T18:42:55.814773+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[31607]: audit 2024-09-10T18:42:55.829943+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]': finished 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[31607]: cluster 2024-09-10T18:42:55.830670+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[31607]: audit 2024-09-10T18:42:55.834161+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[31607]: cluster 2024-09-10T18:42:55.922947+0000 mgr.y (mgr.24512) 348 : cluster [DBG] pgmap v511: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[18597]: cluster 2024-09-10T18:42:55.367564+0000 mon.a (mon.0) 2381 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[18597]: audit 2024-09-10T18:42:55.814773+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[18597]: audit 2024-09-10T18:42:55.829943+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]': finished 2024-09-10T18:42:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[18597]: cluster 2024-09-10T18:42:55.830670+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:42:56.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[18597]: audit 2024-09-10T18:42:55.834161+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:42:56.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:56 smithi100 bash[18597]: cluster 2024-09-10T18:42:55.922947+0000 mgr.y (mgr.24512) 348 : cluster [DBG] pgmap v511: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:56 smithi151 bash[22565]: cluster 2024-09-10T18:42:55.367564+0000 mon.a (mon.0) 2381 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:56 smithi151 bash[22565]: audit 2024-09-10T18:42:55.814773+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:42:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:56 smithi151 bash[22565]: audit 2024-09-10T18:42:55.829943+0000 mon.a (mon.0) 2382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-49"}]': finished 2024-09-10T18:42:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:56 smithi151 bash[22565]: cluster 2024-09-10T18:42:55.830670+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:42:56.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:56 smithi151 bash[22565]: audit 2024-09-10T18:42:55.834161+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:42:56.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:56 smithi151 bash[22565]: cluster 2024-09-10T18:42:55.922947+0000 mgr.y (mgr.24512) 348 : cluster [DBG] pgmap v511: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:57.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[18597]: cluster 2024-09-10T18:42:56.822170+0000 mon.a (mon.0) 2385 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:57.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[18597]: audit 2024-09-10T18:42:56.831689+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]': finished 2024-09-10T18:42:57.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[18597]: cluster 2024-09-10T18:42:56.832002+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:42:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[18597]: audit 2024-09-10T18:42:56.837349+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[31607]: cluster 2024-09-10T18:42:56.822170+0000 mon.a (mon.0) 2385 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[31607]: audit 2024-09-10T18:42:56.831689+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]': finished 2024-09-10T18:42:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[31607]: cluster 2024-09-10T18:42:56.832002+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:42:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:57 smithi100 bash[31607]: audit 2024-09-10T18:42:56.837349+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:57.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:57 smithi151 bash[22565]: cluster 2024-09-10T18:42:56.822170+0000 mon.a (mon.0) 2385 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:57.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:57 smithi151 bash[22565]: audit 2024-09-10T18:42:56.831689+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-49", "mode": "readproxy"}]': finished 2024-09-10T18:42:57.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:57 smithi151 bash[22565]: cluster 2024-09-10T18:42:56.832002+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:42:57.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:57 smithi151 bash[22565]: audit 2024-09-10T18:42:56.837349+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:58 smithi151 bash[22565]: audit 2024-09-10T18:42:57.838812+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:42:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:58 smithi151 bash[22565]: cluster 2024-09-10T18:42:57.838971+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:42:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:58 smithi151 bash[22565]: audit 2024-09-10T18:42:57.840024+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:59.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:58 smithi151 bash[22565]: cluster 2024-09-10T18:42:57.923756+0000 mgr.y (mgr.24512) 349 : cluster [DBG] pgmap v514: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[18597]: audit 2024-09-10T18:42:57.838812+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:42:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[18597]: cluster 2024-09-10T18:42:57.838971+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:42:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[18597]: audit 2024-09-10T18:42:57.840024+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:59.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[18597]: cluster 2024-09-10T18:42:57.923756+0000 mgr.y (mgr.24512) 349 : cluster [DBG] pgmap v514: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[31607]: audit 2024-09-10T18:42:57.838812+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:42:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[31607]: cluster 2024-09-10T18:42:57.838971+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:42:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[31607]: audit 2024-09-10T18:42:57.840024+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]: dispatch 2024-09-10T18:42:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:58 smithi100 bash[31607]: cluster 2024-09-10T18:42:57.923756+0000 mgr.y (mgr.24512) 349 : cluster [DBG] pgmap v514: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: audit 2024-09-10T18:42:58.839291+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: cluster 2024-09-10T18:42:58.839518+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: audit 2024-09-10T18:42:58.863465+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: audit 2024-09-10T18:42:58.863813+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: audit 2024-09-10T18:42:58.864936+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: audit 2024-09-10T18:42:58.865253+0000 mon.a (mon.0) 2395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: audit 2024-09-10T18:42:58.865937+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:42:59 smithi151 bash[22565]: audit 2024-09-10T18:42:58.866323+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:00.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: audit 2024-09-10T18:42:58.839291+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: cluster 2024-09-10T18:42:58.839518+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: audit 2024-09-10T18:42:58.863465+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: audit 2024-09-10T18:42:58.863813+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: audit 2024-09-10T18:42:58.864936+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: audit 2024-09-10T18:42:58.865253+0000 mon.a (mon.0) 2395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: audit 2024-09-10T18:42:58.865937+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[18597]: audit 2024-09-10T18:42:58.866323+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: audit 2024-09-10T18:42:58.839291+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? 172.21.15.100:0/1634584283' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-72601-63"}]': finished 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: cluster 2024-09-10T18:42:58.839518+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:43:00.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: audit 2024-09-10T18:42:58.863465+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: audit 2024-09-10T18:42:58.863813+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: audit 2024-09-10T18:42:58.864936+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: audit 2024-09-10T18:42:58.865253+0000 mon.a (mon.0) 2395 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:00.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: audit 2024-09-10T18:42:58.865937+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:00.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:42:59 smithi100 bash[31607]: audit 2024-09-10T18:42:58.866323+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:00 smithi151 bash[22565]: audit 2024-09-10T18:42:59.856735+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:00 smithi151 bash[22565]: cluster 2024-09-10T18:42:59.856930+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:43:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:00 smithi151 bash[22565]: audit 2024-09-10T18:42:59.859256+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:00 smithi151 bash[22565]: audit 2024-09-10T18:42:59.864477+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:01.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:00 smithi151 bash[22565]: cluster 2024-09-10T18:42:59.924635+0000 mgr.y (mgr.24512) 350 : cluster [DBG] pgmap v517: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[18597]: audit 2024-09-10T18:42:59.856735+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[18597]: cluster 2024-09-10T18:42:59.856930+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:43:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[18597]: audit 2024-09-10T18:42:59.859256+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[18597]: audit 2024-09-10T18:42:59.864477+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:01.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[18597]: cluster 2024-09-10T18:42:59.924635+0000 mgr.y (mgr.24512) 350 : cluster [DBG] pgmap v517: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[31607]: audit 2024-09-10T18:42:59.856735+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-72601-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[31607]: cluster 2024-09-10T18:42:59.856930+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:43:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[31607]: audit 2024-09-10T18:42:59.859256+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[31607]: audit 2024-09-10T18:42:59.864477+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:00 smithi100 bash[31607]: cluster 2024-09-10T18:42:59.924635+0000 mgr.y (mgr.24512) 350 : cluster [DBG] pgmap v517: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:43:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:01] "GET /metrics HTTP/1.1" 200 228513 "" "Prometheus/2.43.0" 2024-09-10T18:43:02.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:01 smithi151 bash[22565]: cluster 2024-09-10T18:43:01.299965+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:43:02.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:01 smithi100 bash[18597]: cluster 2024-09-10T18:43:01.299965+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:43:02.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:01 smithi100 bash[31607]: cluster 2024-09-10T18:43:01.299965+0000 mon.a (mon.0) 2400 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:43:02.679 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:43:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:43:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:02 smithi151 bash[22565]: audit 2024-09-10T18:43:01.883236+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:02 smithi151 bash[22565]: cluster 2024-09-10T18:43:01.883344+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:43:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:02 smithi151 bash[22565]: cluster 2024-09-10T18:43:01.925334+0000 mgr.y (mgr.24512) 351 : cluster [DBG] pgmap v520: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:43:03.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:02 smithi151 bash[22565]: cluster 2024-09-10T18:43:02.883527+0000 mon.a (mon.0) 2403 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:43:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[18597]: audit 2024-09-10T18:43:01.883236+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[18597]: cluster 2024-09-10T18:43:01.883344+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:43:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[18597]: cluster 2024-09-10T18:43:01.925334+0000 mgr.y (mgr.24512) 351 : cluster [DBG] pgmap v520: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:43:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[18597]: cluster 2024-09-10T18:43:02.883527+0000 mon.a (mon.0) 2403 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:43:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[31607]: audit 2024-09-10T18:43:01.883236+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-72601-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[31607]: cluster 2024-09-10T18:43:01.883344+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:43:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[31607]: cluster 2024-09-10T18:43:01.925334+0000 mgr.y (mgr.24512) 351 : cluster [DBG] pgmap v520: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:43:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:02 smithi100 bash[31607]: cluster 2024-09-10T18:43:02.883527+0000 mon.a (mon.0) 2403 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:43:04.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:03 smithi151 bash[22565]: audit 2024-09-10T18:43:02.343872+0000 mgr.y (mgr.24512) 352 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:04.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:03 smithi151 bash[22565]: cluster 2024-09-10T18:43:02.888267+0000 mon.a (mon.0) 2404 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:03 smithi100 bash[18597]: audit 2024-09-10T18:43:02.343872+0000 mgr.y (mgr.24512) 352 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:03 smithi100 bash[18597]: cluster 2024-09-10T18:43:02.888267+0000 mon.a (mon.0) 2404 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:04.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:03 smithi100 bash[31607]: audit 2024-09-10T18:43:02.343872+0000 mgr.y (mgr.24512) 352 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:04.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:03 smithi100 bash[31607]: cluster 2024-09-10T18:43:02.888267+0000 mon.a (mon.0) 2404 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:04.911 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:43:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:04.912 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:04 smithi151 bash[22565]: cluster 2024-09-10T18:43:03.906196+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:43:04.912 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:04 smithi151 bash[22565]: audit 2024-09-10T18:43:03.908427+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:05.178 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:04 smithi151 bash[22565]: audit 2024-09-10T18:43:03.910041+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:05.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:04 smithi151 bash[22565]: cluster 2024-09-10T18:43:03.925932+0000 mgr.y (mgr.24512) 353 : cluster [DBG] pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:43:05.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[31607]: cluster 2024-09-10T18:43:03.906196+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:43:05.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[31607]: audit 2024-09-10T18:43:03.908427+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:05.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[31607]: audit 2024-09-10T18:43:03.910041+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:05.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[31607]: cluster 2024-09-10T18:43:03.925932+0000 mgr.y (mgr.24512) 353 : cluster [DBG] pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:43:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[18597]: cluster 2024-09-10T18:43:03.906196+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:43:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[18597]: audit 2024-09-10T18:43:03.908427+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[18597]: audit 2024-09-10T18:43:03.910041+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:05.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:04 smithi100 bash[18597]: cluster 2024-09-10T18:43:03.925932+0000 mgr.y (mgr.24512) 353 : cluster [DBG] pgmap v523: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:43:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:05 smithi151 bash[22565]: audit 2024-09-10T18:43:04.917975+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:05 smithi151 bash[22565]: cluster 2024-09-10T18:43:04.918351+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:43:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:05 smithi151 bash[22565]: audit 2024-09-10T18:43:04.919418+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:06.179 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:05 smithi151 bash[22565]: audit 2024-09-10T18:43:04.919952+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:06.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[18597]: audit 2024-09-10T18:43:04.917975+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:06.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[18597]: cluster 2024-09-10T18:43:04.918351+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:43:06.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[18597]: audit 2024-09-10T18:43:04.919418+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:06.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[18597]: audit 2024-09-10T18:43:04.919952+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[31607]: audit 2024-09-10T18:43:04.917975+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[31607]: cluster 2024-09-10T18:43:04.918351+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:43:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[31607]: audit 2024-09-10T18:43:04.919418+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.100:0/3702575613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:06.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:05 smithi100 bash[31607]: audit 2024-09-10T18:43:04.919952+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]: dispatch 2024-09-10T18:43:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: audit 2024-09-10T18:43:05.919796+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: cluster 2024-09-10T18:43:05.919966+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:43:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: cluster 2024-09-10T18:43:05.926445+0000 mgr.y (mgr.24512) 354 : cluster [DBG] pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: audit 2024-09-10T18:43:05.948387+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: audit 2024-09-10T18:43:05.948947+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: audit 2024-09-10T18:43:05.950303+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: audit 2024-09-10T18:43:05.950869+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: audit 2024-09-10T18:43:05.951637+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[18597]: audit 2024-09-10T18:43:05.952080+0000 mon.a (mon.0) 2414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: audit 2024-09-10T18:43:05.919796+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: cluster 2024-09-10T18:43:05.919966+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: cluster 2024-09-10T18:43:05.926445+0000 mgr.y (mgr.24512) 354 : cluster [DBG] pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: audit 2024-09-10T18:43:05.948387+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: audit 2024-09-10T18:43:05.948947+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: audit 2024-09-10T18:43:05.950303+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: audit 2024-09-10T18:43:05.950869+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: audit 2024-09-10T18:43:05.951637+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:07.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:06 smithi100 bash[31607]: audit 2024-09-10T18:43:05.952080+0000 mon.a (mon.0) 2414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: audit 2024-09-10T18:43:05.919796+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-72601-64"}]': finished 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: cluster 2024-09-10T18:43:05.919966+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: cluster 2024-09-10T18:43:05.926445+0000 mgr.y (mgr.24512) 354 : cluster [DBG] pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: audit 2024-09-10T18:43:05.948387+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: audit 2024-09-10T18:43:05.948947+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: audit 2024-09-10T18:43:05.950303+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: audit 2024-09-10T18:43:05.950869+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:07.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: audit 2024-09-10T18:43:05.951637+0000 mon.c (mon.2) 428 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:07.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:06 smithi151 bash[22565]: audit 2024-09-10T18:43:05.952080+0000 mon.a (mon.0) 2414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:08 smithi151 bash[22565]: audit 2024-09-10T18:43:06.941905+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:08 smithi151 bash[22565]: cluster 2024-09-10T18:43:06.942010+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:43:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:08 smithi151 bash[22565]: audit 2024-09-10T18:43:06.943175+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:08 smithi151 bash[22565]: audit 2024-09-10T18:43:06.943792+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:08 smithi151 bash[22565]: audit 2024-09-10T18:43:07.446925+0000 mon.b (mon.1) 239 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:08.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:08 smithi151 bash[22565]: audit 2024-09-10T18:43:07.464472+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[18597]: audit 2024-09-10T18:43:06.941905+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[18597]: cluster 2024-09-10T18:43:06.942010+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[18597]: audit 2024-09-10T18:43:06.943175+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[18597]: audit 2024-09-10T18:43:06.943792+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[18597]: audit 2024-09-10T18:43:07.446925+0000 mon.b (mon.1) 239 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[18597]: audit 2024-09-10T18:43:07.464472+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[31607]: audit 2024-09-10T18:43:06.941905+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-72601-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[31607]: cluster 2024-09-10T18:43:06.942010+0000 mon.a (mon.0) 2416 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[31607]: audit 2024-09-10T18:43:06.943175+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[31607]: audit 2024-09-10T18:43:06.943792+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[31607]: audit 2024-09-10T18:43:07.446925+0000 mon.b (mon.1) 239 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:08.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:08 smithi100 bash[31607]: audit 2024-09-10T18:43:07.464472+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: cluster 2024-09-10T18:43:07.927233+0000 mgr.y (mgr.24512) 355 : cluster [DBG] pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.352908+0000 mon.b (mon.1) 240 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.367983+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:43:09.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: cluster 2024-09-10T18:43:08.368172+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:43:09.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.370226+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: cluster 2024-09-10T18:43:08.936430+0000 mon.a (mon.0) 2422 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:09.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.948055+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:09.680 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.948324+0000 mon.a (mon.0) 2424 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]': finished 2024-09-10T18:43:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: cluster 2024-09-10T18:43:08.948419+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:43:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.952890+0000 mon.b (mon.1) 241 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.954106+0000 mon.b (mon.1) 242 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.969985+0000 mon.a (mon.0) 2426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:08.971033+0000 mon.a (mon.0) 2427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:09 smithi151 bash[22565]: audit 2024-09-10T18:43:09.329672+0000 mon.c (mon.2) 430 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: cluster 2024-09-10T18:43:07.927233+0000 mgr.y (mgr.24512) 355 : cluster [DBG] pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.352908+0000 mon.b (mon.1) 240 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.367983+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: cluster 2024-09-10T18:43:08.368172+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.370226+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: cluster 2024-09-10T18:43:08.936430+0000 mon.a (mon.0) 2422 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.948055+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.948324+0000 mon.a (mon.0) 2424 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]': finished 2024-09-10T18:43:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: cluster 2024-09-10T18:43:08.948419+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.952890+0000 mon.b (mon.1) 241 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.954106+0000 mon.b (mon.1) 242 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.969985+0000 mon.a (mon.0) 2426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:08.971033+0000 mon.a (mon.0) 2427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[18597]: audit 2024-09-10T18:43:09.329672+0000 mon.c (mon.2) 430 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: cluster 2024-09-10T18:43:07.927233+0000 mgr.y (mgr.24512) 355 : cluster [DBG] pgmap v528: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.352908+0000 mon.b (mon.1) 240 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.367983+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: cluster 2024-09-10T18:43:08.368172+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.370226+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: cluster 2024-09-10T18:43:08.936430+0000 mon.a (mon.0) 2422 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.948055+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-72601-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.948324+0000 mon.a (mon.0) 2424 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]': finished 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: cluster 2024-09-10T18:43:08.948419+0000 mon.a (mon.0) 2425 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.952890+0000 mon.b (mon.1) 241 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.954106+0000 mon.b (mon.1) 242 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.969985+0000 mon.a (mon.0) 2426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:08.971033+0000 mon.a (mon.0) 2427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-49"}]: dispatch 2024-09-10T18:43:09.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:09 smithi100 bash[31607]: audit 2024-09-10T18:43:09.329672+0000 mon.c (mon.2) 430 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:09.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:7462ddf6:::163:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:5d165639:::164:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f43765fc:::165:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b4c720e9:::166:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:e694b040:::167:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:afa38db2:::168:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:77ba9f53:::169:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:87495034:::170:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:7c96bf0e:::171:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:dbe346cc:::172:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:e943ec24:::173:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f97a9c0c:::174:head 2024-09-10T18:43:09.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:6f26e74d:::175:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:4f95e106:::176:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:0e6f2f8f:::177:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:05db05f1:::178:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:38a78d66:::179:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:d095610b:::180:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:a1a9d709:::181:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:1e5d39db:::182:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:f7df4fb9:::183:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:03a7f161:::184:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:ba70721e:::185:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:28e5662d:::186:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:973d52de:::187:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:4303eb1c:::188:head 2024-09-10T18:43:09.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:b990b48e:::189:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:29b8165b:::190:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:3547f197:::191:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:7e260936:::192:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:1abec7b1:::193:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:10fdda93:::194:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:15817eea:::195:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:770bab57:::196:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:ed9e13e7:::197:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:71471a8f:::198:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 195:10fb1d02:::199:head 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (7651 ms) 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,0 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first is 1725993744 2024-09-10T18:43:09.954 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,1725993749,1725993750,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,1725993749,1725993750,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,1725993749,1725993750,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,1725993749,1725993750,1725993752,1725993753,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,1725993749,1725993750,1725993752,1725993753,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993744,1725993746,1725993747,1725993749,1725993750,1725993752,1725993753,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725993747,1725993749,1725993750,1725993752,1725993753,1725993755,1725993756,0 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first now 1725993747, trimmed 2024-09-10T18:43:09.955 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21141 ms) 2024-09-10T18:43:09.956 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-10T18:43:09.956 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: foo0 2024-09-10T18:43:09.956 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-10T18:43:09.956 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14120 ms) 2024-09-10T18:43:09.956 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-10T18:43:11.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:10 smithi100 bash[18597]: cluster 2024-09-10T18:43:09.928055+0000 mgr.y (mgr.24512) 356 : cluster [DBG] pgmap v531: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:11.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:10 smithi100 bash[18597]: cluster 2024-09-10T18:43:09.948715+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:43:11.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:10 smithi100 bash[18597]: cluster 2024-09-10T18:43:10.381952+0000 mon.a (mon.0) 2429 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:11.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:10 smithi100 bash[31607]: cluster 2024-09-10T18:43:09.928055+0000 mgr.y (mgr.24512) 356 : cluster [DBG] pgmap v531: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:11.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:10 smithi100 bash[31607]: cluster 2024-09-10T18:43:09.948715+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:43:11.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:10 smithi100 bash[31607]: cluster 2024-09-10T18:43:10.381952+0000 mon.a (mon.0) 2429 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:11.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:10 smithi151 bash[22565]: cluster 2024-09-10T18:43:09.928055+0000 mgr.y (mgr.24512) 356 : cluster [DBG] pgmap v531: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:11.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:10 smithi151 bash[22565]: cluster 2024-09-10T18:43:09.948715+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:43:11.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:10 smithi151 bash[22565]: cluster 2024-09-10T18:43:10.381952+0000 mon.a (mon.0) 2429 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:11] "GET /metrics HTTP/1.1" 200 230911 "" "Prometheus/2.43.0" 2024-09-10T18:43:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[18597]: audit 2024-09-10T18:43:10.939636+0000 mon.b (mon.1) 243 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[18597]: cluster 2024-09-10T18:43:10.953791+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:43:12.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[18597]: audit 2024-09-10T18:43:10.965325+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:12.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[18597]: audit 2024-09-10T18:43:10.969008+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:12.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[18597]: audit 2024-09-10T18:43:10.969551+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[31607]: audit 2024-09-10T18:43:10.939636+0000 mon.b (mon.1) 243 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[31607]: cluster 2024-09-10T18:43:10.953791+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:43:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[31607]: audit 2024-09-10T18:43:10.965325+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[31607]: audit 2024-09-10T18:43:10.969008+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:12.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:11 smithi100 bash[31607]: audit 2024-09-10T18:43:10.969551+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:12.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:11 smithi151 bash[22565]: audit 2024-09-10T18:43:10.939636+0000 mon.b (mon.1) 243 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:12.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:11 smithi151 bash[22565]: cluster 2024-09-10T18:43:10.953791+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:43:12.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:11 smithi151 bash[22565]: audit 2024-09-10T18:43:10.965325+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:12.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:11 smithi151 bash[22565]: audit 2024-09-10T18:43:10.969008+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:12.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:11 smithi151 bash[22565]: audit 2024-09-10T18:43:10.969551+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:12.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: cluster 2024-09-10T18:43:11.928886+0000 mgr.y (mgr.24512) 357 : cluster [DBG] pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:11.957327+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:11.957983+0000 mon.a (mon.0) 2434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: cluster 2024-09-10T18:43:11.958319+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:11.960026+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:11.962427+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:11.969104+0000 mon.b (mon.1) 244 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:11.985995+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:13.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:12.944177+0000 mon.b (mon.1) 245 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:12.958538+0000 mon.a (mon.0) 2438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:12.958650+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: cluster 2024-09-10T18:43:12.958749+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[18597]: audit 2024-09-10T18:43:12.963019+0000 mon.a (mon.0) 2441 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: cluster 2024-09-10T18:43:11.928886+0000 mgr.y (mgr.24512) 357 : cluster [DBG] pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:11.957327+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:11.957983+0000 mon.a (mon.0) 2434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:13.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: cluster 2024-09-10T18:43:11.958319+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:43:13.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:11.960026+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:13.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:11.962427+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:13.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:11.969104+0000 mon.b (mon.1) 244 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:13.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:11.985995+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:13.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:12.944177+0000 mon.b (mon.1) 245 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:13.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:12.958538+0000 mon.a (mon.0) 2438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:13.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:12.958650+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:13.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: cluster 2024-09-10T18:43:12.958749+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:43:13.297 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:12 smithi100 bash[31607]: audit 2024-09-10T18:43:12.963019+0000 mon.a (mon.0) 2441 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: cluster 2024-09-10T18:43:11.928886+0000 mgr.y (mgr.24512) 357 : cluster [DBG] pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:11.957327+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:11.957983+0000 mon.a (mon.0) 2434 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: cluster 2024-09-10T18:43:11.958319+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:11.960026+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.100:0/2697932137' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:11.962427+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]: dispatch 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:11.969104+0000 mon.b (mon.1) 244 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:13.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:11.985995+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:13.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:12.944177+0000 mon.b (mon.1) 245 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:13.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:12.958538+0000 mon.a (mon.0) 2438 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-72601-65"}]': finished 2024-09-10T18:43:13.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:12.958650+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:13.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: cluster 2024-09-10T18:43:12.958749+0000 mon.a (mon.0) 2440 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:43:13.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:12 smithi151 bash[22565]: audit 2024-09-10T18:43:12.963019+0000 mon.a (mon.0) 2441 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:12.354968+0000 mgr.y (mgr.24512) 358 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:12.985520+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:12.986961+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:12.987440+0000 mon.a (mon.0) 2444 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-72601-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:13.946623+0000 mon.b (mon.1) 246 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]: dispatch 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:13.961185+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]': finished 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:13.961341+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-72601-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: cluster 2024-09-10T18:43:13.961458+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:13.962955+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-72601-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[18597]: audit 2024-09-10T18:43:13.963744+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]: dispatch 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:12.354968+0000 mgr.y (mgr.24512) 358 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:12.985520+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:12.986961+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:12.987440+0000 mon.a (mon.0) 2444 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-72601-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:13.946623+0000 mon.b (mon.1) 246 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]: dispatch 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:13.961185+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]': finished 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:13.961341+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-72601-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: cluster 2024-09-10T18:43:13.961458+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:13.962955+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-72601-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:13 smithi100 bash[31607]: audit 2024-09-10T18:43:13.963744+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]: dispatch 2024-09-10T18:43:14.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:12.354968+0000 mgr.y (mgr.24512) 358 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:12.985520+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:12.986961+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:12.987440+0000 mon.a (mon.0) 2444 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-72601-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:13.946623+0000 mon.b (mon.1) 246 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]: dispatch 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:13.961185+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-51"}]': finished 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:13.961341+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-72601-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: cluster 2024-09-10T18:43:13.961458+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:13.962955+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-72601-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:14.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:13 smithi151 bash[22565]: audit 2024-09-10T18:43:13.963744+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]: dispatch 2024-09-10T18:43:14.928 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:43:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:15.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:15 smithi151 bash[22565]: cluster 2024-09-10T18:43:13.929772+0000 mgr.y (mgr.24512) 359 : cluster [DBG] pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:15.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:15 smithi151 bash[22565]: cluster 2024-09-10T18:43:14.955845+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:15 smithi100 bash[18597]: cluster 2024-09-10T18:43:13.929772+0000 mgr.y (mgr.24512) 359 : cluster [DBG] pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:15 smithi100 bash[18597]: cluster 2024-09-10T18:43:14.955845+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:15 smithi100 bash[31607]: cluster 2024-09-10T18:43:13.929772+0000 mgr.y (mgr.24512) 359 : cluster [DBG] pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:15 smithi100 bash[31607]: cluster 2024-09-10T18:43:14.955845+0000 mon.a (mon.0) 2450 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: audit 2024-09-10T18:43:15.393889+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]': finished 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: cluster 2024-09-10T18:43:15.394106+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: audit 2024-09-10T18:43:15.450162+0000 mon.b (mon.1) 247 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: audit 2024-09-10T18:43:15.467327+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: cluster 2024-09-10T18:43:15.931820+0000 mgr.y (mgr.24512) 360 : cluster [DBG] pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: audit 2024-09-10T18:43:15.955971+0000 mon.b (mon.1) 248 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: audit 2024-09-10T18:43:15.968949+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-72601-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: audit 2024-09-10T18:43:15.969160+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: cluster 2024-09-10T18:43:15.969306+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:43:16.679 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:16 smithi151 bash[22565]: audit 2024-09-10T18:43:15.976281+0000 mon.a (mon.0) 2457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:43:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: audit 2024-09-10T18:43:15.393889+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]': finished 2024-09-10T18:43:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: cluster 2024-09-10T18:43:15.394106+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: audit 2024-09-10T18:43:15.450162+0000 mon.b (mon.1) 247 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: audit 2024-09-10T18:43:15.467327+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: cluster 2024-09-10T18:43:15.931820+0000 mgr.y (mgr.24512) 360 : cluster [DBG] pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: audit 2024-09-10T18:43:15.955971+0000 mon.b (mon.1) 248 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: audit 2024-09-10T18:43:15.968949+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-72601-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: audit 2024-09-10T18:43:15.969160+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: cluster 2024-09-10T18:43:15.969306+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[18597]: audit 2024-09-10T18:43:15.976281+0000 mon.a (mon.0) 2457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: audit 2024-09-10T18:43:15.393889+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-51", "mode": "writeback"}]': finished 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: cluster 2024-09-10T18:43:15.394106+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: audit 2024-09-10T18:43:15.450162+0000 mon.b (mon.1) 247 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:43:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: audit 2024-09-10T18:43:15.467327+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:43:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: cluster 2024-09-10T18:43:15.931820+0000 mgr.y (mgr.24512) 360 : cluster [DBG] pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: audit 2024-09-10T18:43:15.955971+0000 mon.b (mon.1) 248 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:43:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: audit 2024-09-10T18:43:15.968949+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-72601-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: audit 2024-09-10T18:43:15.969160+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:43:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: cluster 2024-09-10T18:43:15.969306+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:43:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:16 smithi100 bash[31607]: audit 2024-09-10T18:43:15.976281+0000 mon.a (mon.0) 2457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:43:18.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[18597]: audit 2024-09-10T18:43:16.959757+0000 mon.b (mon.1) 249 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:43:18.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[18597]: audit 2024-09-10T18:43:16.971942+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:43:18.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[18597]: cluster 2024-09-10T18:43:16.972197+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:43:18.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[18597]: audit 2024-09-10T18:43:16.976934+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:43:18.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[31607]: audit 2024-09-10T18:43:16.959757+0000 mon.b (mon.1) 249 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:43:18.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[31607]: audit 2024-09-10T18:43:16.971942+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:43:18.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[31607]: cluster 2024-09-10T18:43:16.972197+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:43:18.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:17 smithi100 bash[31607]: audit 2024-09-10T18:43:16.976934+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:43:18.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:17 smithi151 bash[22565]: audit 2024-09-10T18:43:16.959757+0000 mon.b (mon.1) 249 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:43:18.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:17 smithi151 bash[22565]: audit 2024-09-10T18:43:16.971942+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:43:18.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:17 smithi151 bash[22565]: cluster 2024-09-10T18:43:16.972197+0000 mon.a (mon.0) 2459 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:43:18.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:17 smithi151 bash[22565]: audit 2024-09-10T18:43:16.976934+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:43:19.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: cluster 2024-09-10T18:43:17.932675+0000 mgr.y (mgr.24512) 361 : cluster [DBG] pgmap v543: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: audit 2024-09-10T18:43:17.962200+0000 mon.b (mon.1) 250 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: cluster 2024-09-10T18:43:17.965824+0000 mon.a (mon.0) 2461 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: cluster 2024-09-10T18:43:17.966501+0000 mon.a (mon.0) 2462 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: audit 2024-09-10T18:43:17.976208+0000 mon.a (mon.0) 2463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: cluster 2024-09-10T18:43:17.976517+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: audit 2024-09-10T18:43:17.979682+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: audit 2024-09-10T18:43:17.986784+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: audit 2024-09-10T18:43:18.977716+0000 mon.a (mon.0) 2467 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: audit 2024-09-10T18:43:18.977922+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: cluster 2024-09-10T18:43:18.978131+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:43:19.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[18597]: audit 2024-09-10T18:43:18.979949+0000 mon.a (mon.0) 2470 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: cluster 2024-09-10T18:43:17.932675+0000 mgr.y (mgr.24512) 361 : cluster [DBG] pgmap v543: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: audit 2024-09-10T18:43:17.962200+0000 mon.b (mon.1) 250 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: cluster 2024-09-10T18:43:17.965824+0000 mon.a (mon.0) 2461 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: cluster 2024-09-10T18:43:17.966501+0000 mon.a (mon.0) 2462 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: audit 2024-09-10T18:43:17.976208+0000 mon.a (mon.0) 2463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: cluster 2024-09-10T18:43:17.976517+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: audit 2024-09-10T18:43:17.979682+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: audit 2024-09-10T18:43:17.986784+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: audit 2024-09-10T18:43:18.977716+0000 mon.a (mon.0) 2467 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: audit 2024-09-10T18:43:18.977922+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:43:19.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: cluster 2024-09-10T18:43:18.978131+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:43:19.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:18 smithi100 bash[31607]: audit 2024-09-10T18:43:18.979949+0000 mon.a (mon.0) 2470 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: cluster 2024-09-10T18:43:17.932675+0000 mgr.y (mgr.24512) 361 : cluster [DBG] pgmap v543: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: audit 2024-09-10T18:43:17.962200+0000 mon.b (mon.1) 250 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: cluster 2024-09-10T18:43:17.965824+0000 mon.a (mon.0) 2461 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: cluster 2024-09-10T18:43:17.966501+0000 mon.a (mon.0) 2462 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: audit 2024-09-10T18:43:17.976208+0000 mon.a (mon.0) 2463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: cluster 2024-09-10T18:43:17.976517+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: audit 2024-09-10T18:43:17.979682+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: audit 2024-09-10T18:43:17.986784+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: audit 2024-09-10T18:43:18.977716+0000 mon.a (mon.0) 2467 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:19.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: audit 2024-09-10T18:43:18.977922+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:43:19.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: cluster 2024-09-10T18:43:18.978131+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:43:19.430 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:18 smithi151 bash[22565]: audit 2024-09-10T18:43:18.979949+0000 mon.a (mon.0) 2470 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]: dispatch 2024-09-10T18:43:19.985 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ Oapi_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7067 ms) 2024-09-10T18:43:19.985 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-10T18:43:19.985 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7037 ms) 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7044 ms) 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7053 ms) 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3036 ms) 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7070 ms) 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-10T18:43:19.986 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7042 ms) 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7044 ms) 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7087 ms) 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7033 ms) 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7020 ms) 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139590 ms total) 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-09-10T18:43:19.987 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-10T18:43:19.988 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (305689 ms total) 2024-09-10T18:43:19.988 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-10T18:43:20.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[18597]: audit 2024-09-10T18:43:18.965258+0000 mon.b (mon.1) 251 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:43:20.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[18597]: audit 2024-09-10T18:43:18.982178+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:43:20.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[18597]: audit 2024-09-10T18:43:19.979781+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:20.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[18597]: audit 2024-09-10T18:43:19.979900+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:43:20.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[18597]: cluster 2024-09-10T18:43:19.979985+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:43:20.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[31607]: audit 2024-09-10T18:43:18.965258+0000 mon.b (mon.1) 251 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:43:20.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[31607]: audit 2024-09-10T18:43:18.982178+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:43:20.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[31607]: audit 2024-09-10T18:43:19.979781+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:20.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[31607]: audit 2024-09-10T18:43:19.979900+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:43:20.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:19 smithi100 bash[31607]: cluster 2024-09-10T18:43:19.979985+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:43:20.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:19 smithi151 bash[22565]: audit 2024-09-10T18:43:18.965258+0000 mon.b (mon.1) 251 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:43:20.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:19 smithi151 bash[22565]: audit 2024-09-10T18:43:18.982178+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:43:20.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:19 smithi151 bash[22565]: audit 2024-09-10T18:43:19.979781+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? 172.21.15.100:0/3050494871' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-72601-66"}]': finished 2024-09-10T18:43:20.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:19 smithi151 bash[22565]: audit 2024-09-10T18:43:19.979900+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:43:20.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:19 smithi151 bash[22565]: cluster 2024-09-10T18:43:19.979985+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:43:21.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:20 smithi100 bash[31607]: cluster 2024-09-10T18:43:19.933596+0000 mgr.y (mgr.24512) 362 : cluster [DBG] pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:21.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:20 smithi100 bash[18597]: cluster 2024-09-10T18:43:19.933596+0000 mgr.y (mgr.24512) 362 : cluster [DBG] pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:21.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:20 smithi151 bash[22565]: cluster 2024-09-10T18:43:19.933596+0000 mgr.y (mgr.24512) 362 : cluster [DBG] pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:43:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:21] "GET /metrics HTTP/1.1" 200 230911 "" "Prometheus/2.43.0" 2024-09-10T18:43:22.360 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:22 smithi151 bash[22565]: cluster 2024-09-10T18:43:21.935817+0000 mgr.y (mgr.24512) 363 : cluster [DBG] pgmap v548: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:43:22.361 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:22 smithi151 bash[22565]: cluster 2024-09-10T18:43:21.995045+0000 mon.a (mon.0) 2475 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:43:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:22 smithi100 bash[18597]: cluster 2024-09-10T18:43:21.935817+0000 mgr.y (mgr.24512) 363 : cluster [DBG] pgmap v548: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:43:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:22 smithi100 bash[18597]: cluster 2024-09-10T18:43:21.995045+0000 mon.a (mon.0) 2475 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:43:22.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:22 smithi100 bash[31607]: cluster 2024-09-10T18:43:21.935817+0000 mgr.y (mgr.24512) 363 : cluster [DBG] pgmap v548: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:43:22.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:22 smithi100 bash[31607]: cluster 2024-09-10T18:43:21.995045+0000 mon.a (mon.0) 2475 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:43:22.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:43:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:43:23.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:23 smithi151 bash[22565]: audit 2024-09-10T18:43:22.360451+0000 mgr.y (mgr.24512) 364 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:23.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:23 smithi100 bash[18597]: audit 2024-09-10T18:43:22.360451+0000 mgr.y (mgr.24512) 364 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:23.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:23 smithi100 bash[31607]: audit 2024-09-10T18:43:22.360451+0000 mgr.y (mgr.24512) 364 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:24.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:24 smithi151 bash[22565]: cluster 2024-09-10T18:43:23.936658+0000 mgr.y (mgr.24512) 365 : cluster [DBG] pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:24.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:24 smithi100 bash[31607]: cluster 2024-09-10T18:43:23.936658+0000 mgr.y (mgr.24512) 365 : cluster [DBG] pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:24.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:24 smithi100 bash[18597]: cluster 2024-09-10T18:43:23.936658+0000 mgr.y (mgr.24512) 365 : cluster [DBG] pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:43:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:25 smithi151 bash[22565]: audit 2024-09-10T18:43:24.337329+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:25.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:25 smithi151 bash[22565]: audit 2024-09-10T18:43:24.339954+0000 mon.c (mon.2) 433 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:25.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:25 smithi100 bash[31607]: audit 2024-09-10T18:43:24.337329+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:25.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:25 smithi100 bash[31607]: audit 2024-09-10T18:43:24.339954+0000 mon.c (mon.2) 433 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:25 smithi100 bash[18597]: audit 2024-09-10T18:43:24.337329+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:25 smithi100 bash[18597]: audit 2024-09-10T18:43:24.339954+0000 mon.c (mon.2) 433 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:26 smithi151 bash[22565]: cluster 2024-09-10T18:43:25.400531+0000 mon.a (mon.0) 2477 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:26.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:26 smithi151 bash[22565]: cluster 2024-09-10T18:43:25.938958+0000 mgr.y (mgr.24512) 366 : cluster [DBG] pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:26.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:26 smithi100 bash[31607]: cluster 2024-09-10T18:43:25.400531+0000 mon.a (mon.0) 2477 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:26.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:26 smithi100 bash[31607]: cluster 2024-09-10T18:43:25.938958+0000 mgr.y (mgr.24512) 366 : cluster [DBG] pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:26 smithi100 bash[18597]: cluster 2024-09-10T18:43:25.400531+0000 mon.a (mon.0) 2477 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:26.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:26 smithi100 bash[18597]: cluster 2024-09-10T18:43:25.938958+0000 mgr.y (mgr.24512) 366 : cluster [DBG] pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:29.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:28 smithi100 bash[18597]: cluster 2024-09-10T18:43:27.939806+0000 mgr.y (mgr.24512) 367 : cluster [DBG] pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:29.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:29 smithi100 bash[31607]: cluster 2024-09-10T18:43:27.939806+0000 mgr.y (mgr.24512) 367 : cluster [DBG] pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:29.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:28 smithi151 bash[22565]: cluster 2024-09-10T18:43:27.939806+0000 mgr.y (mgr.24512) 367 : cluster [DBG] pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:31 smithi100 bash[18597]: cluster 2024-09-10T18:43:29.940713+0000 mgr.y (mgr.24512) 368 : cluster [DBG] pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:31 smithi100 bash[18597]: audit 2024-09-10T18:43:29.993536+0000 mon.b (mon.1) 252 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:31.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:31 smithi100 bash[18597]: audit 2024-09-10T18:43:30.010504+0000 mon.a (mon.0) 2478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:31 smithi100 bash[31607]: cluster 2024-09-10T18:43:29.940713+0000 mgr.y (mgr.24512) 368 : cluster [DBG] pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:31 smithi100 bash[31607]: audit 2024-09-10T18:43:29.993536+0000 mon.b (mon.1) 252 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:31.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:31 smithi100 bash[31607]: audit 2024-09-10T18:43:30.010504+0000 mon.a (mon.0) 2478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:30 smithi151 bash[22565]: cluster 2024-09-10T18:43:29.940713+0000 mgr.y (mgr.24512) 368 : cluster [DBG] pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:30 smithi151 bash[22565]: audit 2024-09-10T18:43:29.993536+0000 mon.b (mon.1) 252 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:30 smithi151 bash[22565]: audit 2024-09-10T18:43:30.010504+0000 mon.a (mon.0) 2478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:43:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:31] "GET /metrics HTTP/1.1" 200 228632 "" "Prometheus/2.43.0" 2024-09-10T18:43:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[18597]: audit 2024-09-10T18:43:30.993949+0000 mon.b (mon.1) 253 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:32.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[18597]: audit 2024-09-10T18:43:31.008114+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[18597]: cluster 2024-09-10T18:43:31.008333+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[18597]: audit 2024-09-10T18:43:31.011873+0000 mon.a (mon.0) 2481 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[18597]: audit 2024-09-10T18:43:32.011046+0000 mon.a (mon.0) 2482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]': finished 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[18597]: cluster 2024-09-10T18:43:32.011203+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[31607]: audit 2024-09-10T18:43:30.993949+0000 mon.b (mon.1) 253 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[31607]: audit 2024-09-10T18:43:31.008114+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[31607]: cluster 2024-09-10T18:43:31.008333+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:43:32.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[31607]: audit 2024-09-10T18:43:31.011873+0000 mon.a (mon.0) 2481 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[31607]: audit 2024-09-10T18:43:32.011046+0000 mon.a (mon.0) 2482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]': finished 2024-09-10T18:43:32.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:32 smithi100 bash[31607]: cluster 2024-09-10T18:43:32.011203+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:43:32.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:32 smithi151 bash[22565]: audit 2024-09-10T18:43:30.993949+0000 mon.b (mon.1) 253 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:32.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:32 smithi151 bash[22565]: audit 2024-09-10T18:43:31.008114+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:43:32.372 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:32 smithi151 bash[22565]: cluster 2024-09-10T18:43:31.008333+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:43:32.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:32 smithi151 bash[22565]: audit 2024-09-10T18:43:31.011873+0000 mon.a (mon.0) 2481 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:32.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:32 smithi151 bash[22565]: audit 2024-09-10T18:43:32.011046+0000 mon.a (mon.0) 2482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]': finished 2024-09-10T18:43:32.373 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:32 smithi151 bash[22565]: cluster 2024-09-10T18:43:32.011203+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:43:32.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:43:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:43:33.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[18597]: cluster 2024-09-10T18:43:31.941589+0000 mgr.y (mgr.24512) 369 : cluster [DBG] pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:43:33.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[18597]: audit 2024-09-10T18:43:32.040300+0000 mon.b (mon.1) 254 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:33.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[18597]: audit 2024-09-10T18:43:32.042037+0000 mon.b (mon.1) 255 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:33.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[18597]: audit 2024-09-10T18:43:32.056808+0000 mon.a (mon.0) 2484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:33.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[18597]: audit 2024-09-10T18:43:32.058485+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[31607]: cluster 2024-09-10T18:43:31.941589+0000 mgr.y (mgr.24512) 369 : cluster [DBG] pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:43:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[31607]: audit 2024-09-10T18:43:32.040300+0000 mon.b (mon.1) 254 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[31607]: audit 2024-09-10T18:43:32.042037+0000 mon.b (mon.1) 255 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[31607]: audit 2024-09-10T18:43:32.056808+0000 mon.a (mon.0) 2484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:33.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:33 smithi100 bash[31607]: audit 2024-09-10T18:43:32.058485+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:33.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:33 smithi151 bash[22565]: cluster 2024-09-10T18:43:31.941589+0000 mgr.y (mgr.24512) 369 : cluster [DBG] pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:43:33.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:33 smithi151 bash[22565]: audit 2024-09-10T18:43:32.040300+0000 mon.b (mon.1) 254 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:33.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:33 smithi151 bash[22565]: audit 2024-09-10T18:43:32.042037+0000 mon.b (mon.1) 255 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:33.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:33 smithi151 bash[22565]: audit 2024-09-10T18:43:32.056808+0000 mon.a (mon.0) 2484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:33.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:33 smithi151 bash[22565]: audit 2024-09-10T18:43:32.058485+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-51"}]: dispatch 2024-09-10T18:43:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:34 smithi151 bash[22565]: audit 2024-09-10T18:43:32.371618+0000 mgr.y (mgr.24512) 370 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:34.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:34 smithi151 bash[22565]: cluster 2024-09-10T18:43:33.041749+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:43:34.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:34 smithi100 bash[31607]: audit 2024-09-10T18:43:32.371618+0000 mgr.y (mgr.24512) 370 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:34.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:34 smithi100 bash[31607]: cluster 2024-09-10T18:43:33.041749+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:43:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:34 smithi100 bash[18597]: audit 2024-09-10T18:43:32.371618+0000 mgr.y (mgr.24512) 370 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:34 smithi100 bash[18597]: cluster 2024-09-10T18:43:33.041749+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:43:34.928 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:43:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:35 smithi151 bash[22565]: cluster 2024-09-10T18:43:33.942374+0000 mgr.y (mgr.24512) 371 : cluster [DBG] pgmap v557: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:35 smithi151 bash[22565]: cluster 2024-09-10T18:43:34.037931+0000 mon.a (mon.0) 2487 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:35 smithi151 bash[22565]: audit 2024-09-10T18:43:34.047644+0000 mon.b (mon.1) 256 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:35 smithi151 bash[22565]: cluster 2024-09-10T18:43:34.061684+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:35 smithi151 bash[22565]: audit 2024-09-10T18:43:34.064408+0000 mon.a (mon.0) 2489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[18597]: cluster 2024-09-10T18:43:33.942374+0000 mgr.y (mgr.24512) 371 : cluster [DBG] pgmap v557: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T18:43:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[18597]: cluster 2024-09-10T18:43:34.037931+0000 mon.a (mon.0) 2487 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:43:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[18597]: audit 2024-09-10T18:43:34.047644+0000 mon.b (mon.1) 256 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[18597]: cluster 2024-09-10T18:43:34.061684+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:43:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[18597]: audit 2024-09-10T18:43:34.064408+0000 mon.a (mon.0) 2489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[31607]: cluster 2024-09-10T18:43:33.942374+0000 mgr.y (mgr.24512) 371 : cluster [DBG] pgmap v557: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T18:43:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[31607]: cluster 2024-09-10T18:43:34.037931+0000 mon.a (mon.0) 2487 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:43:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[31607]: audit 2024-09-10T18:43:34.047644+0000 mon.b (mon.1) 256 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[31607]: cluster 2024-09-10T18:43:34.061684+0000 mon.a (mon.0) 2488 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:43:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:35 smithi100 bash[31607]: audit 2024-09-10T18:43:34.064408+0000 mon.a (mon.0) 2489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:36.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:36 smithi151 bash[22565]: audit 2024-09-10T18:43:35.063791+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:36.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:36 smithi151 bash[22565]: cluster 2024-09-10T18:43:35.064261+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:43:36.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:36 smithi151 bash[22565]: audit 2024-09-10T18:43:35.093215+0000 mon.b (mon.1) 257 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:36.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:36 smithi151 bash[22565]: audit 2024-09-10T18:43:35.094668+0000 mon.b (mon.1) 258 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-53"}]: dispatch 2024-09-10T18:43:36.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:36 smithi151 bash[22565]: audit 2024-09-10T18:43:35.109583+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:36.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:36 smithi151 bash[22565]: audit 2024-09-10T18:43:35.111003+0000 mon.a (mon.0) 2493 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-53"}]: dispatch 2024-09-10T18:43:36.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:36 smithi151 bash[22565]: cluster 2024-09-10T18:43:35.944479+0000 mgr.y (mgr.24512) 372 : cluster [DBG] pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:43:36.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[18597]: audit 2024-09-10T18:43:35.063791+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:36.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[18597]: cluster 2024-09-10T18:43:35.064261+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[18597]: audit 2024-09-10T18:43:35.093215+0000 mon.b (mon.1) 257 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[18597]: audit 2024-09-10T18:43:35.094668+0000 mon.b (mon.1) 258 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-53"}]: dispatch 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[18597]: audit 2024-09-10T18:43:35.109583+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[18597]: audit 2024-09-10T18:43:35.111003+0000 mon.a (mon.0) 2493 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-53"}]: dispatch 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[18597]: cluster 2024-09-10T18:43:35.944479+0000 mgr.y (mgr.24512) 372 : cluster [DBG] pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[31607]: audit 2024-09-10T18:43:35.063791+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[31607]: cluster 2024-09-10T18:43:35.064261+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[31607]: audit 2024-09-10T18:43:35.093215+0000 mon.b (mon.1) 257 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[31607]: audit 2024-09-10T18:43:35.094668+0000 mon.b (mon.1) 258 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-53"}]: dispatch 2024-09-10T18:43:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[31607]: audit 2024-09-10T18:43:35.109583+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[31607]: audit 2024-09-10T18:43:35.111003+0000 mon.a (mon.0) 2493 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-53"}]: dispatch 2024-09-10T18:43:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:36 smithi100 bash[31607]: cluster 2024-09-10T18:43:35.944479+0000 mgr.y (mgr.24512) 372 : cluster [DBG] pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:43:37.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:37 smithi151 bash[22565]: cluster 2024-09-10T18:43:36.080396+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:43:37.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:37 smithi100 bash[18597]: cluster 2024-09-10T18:43:36.080396+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:43:37.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:37 smithi100 bash[31607]: cluster 2024-09-10T18:43:36.080396+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: audit 2024-09-10T18:43:37.091287+0000 mon.b (mon.1) 259 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: cluster 2024-09-10T18:43:37.104630+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: audit 2024-09-10T18:43:37.108209+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: audit 2024-09-10T18:43:37.430687+0000 mon.c (mon.2) 434 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: audit 2024-09-10T18:43:37.878195+0000 mon.c (mon.2) 435 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: audit 2024-09-10T18:43:37.879714+0000 mon.c (mon.2) 436 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: audit 2024-09-10T18:43:37.889959+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:38.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:38 smithi151 bash[22565]: cluster 2024-09-10T18:43:37.945245+0000 mgr.y (mgr.24512) 373 : cluster [DBG] pgmap v563: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:38.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: audit 2024-09-10T18:43:37.091287+0000 mon.b (mon.1) 259 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: cluster 2024-09-10T18:43:37.104630+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: audit 2024-09-10T18:43:37.108209+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: audit 2024-09-10T18:43:37.430687+0000 mon.c (mon.2) 434 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: audit 2024-09-10T18:43:37.878195+0000 mon.c (mon.2) 435 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: audit 2024-09-10T18:43:37.879714+0000 mon.c (mon.2) 436 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: audit 2024-09-10T18:43:37.889959+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[18597]: cluster 2024-09-10T18:43:37.945245+0000 mgr.y (mgr.24512) 373 : cluster [DBG] pgmap v563: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: audit 2024-09-10T18:43:37.091287+0000 mon.b (mon.1) 259 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: cluster 2024-09-10T18:43:37.104630+0000 mon.a (mon.0) 2495 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:43:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: audit 2024-09-10T18:43:37.108209+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: audit 2024-09-10T18:43:37.430687+0000 mon.c (mon.2) 434 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: audit 2024-09-10T18:43:37.878195+0000 mon.c (mon.2) 435 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: audit 2024-09-10T18:43:37.879714+0000 mon.c (mon.2) 436 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: audit 2024-09-10T18:43:37.889959+0000 mon.a (mon.0) 2497 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:38 smithi100 bash[31607]: cluster 2024-09-10T18:43:37.945245+0000 mgr.y (mgr.24512) 373 : cluster [DBG] pgmap v563: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:39.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: audit 2024-09-10T18:43:38.107113+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:39.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: audit 2024-09-10T18:43:38.117229+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:39.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: cluster 2024-09-10T18:43:38.117447+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:43:39.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: audit 2024-09-10T18:43:38.161422+0000 mon.b (mon.1) 261 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:39.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: audit 2024-09-10T18:43:38.162991+0000 mon.b (mon.1) 262 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-55"}]: dispatch 2024-09-10T18:43:39.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: audit 2024-09-10T18:43:38.177729+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:39.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: audit 2024-09-10T18:43:38.179182+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-55"}]: dispatch 2024-09-10T18:43:39.929 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:39 smithi151 bash[22565]: cluster 2024-09-10T18:43:38.886001+0000 mon.a (mon.0) 2502 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:40.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: audit 2024-09-10T18:43:38.107113+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:40.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: audit 2024-09-10T18:43:38.117229+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: cluster 2024-09-10T18:43:38.117447+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: audit 2024-09-10T18:43:38.161422+0000 mon.b (mon.1) 261 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: audit 2024-09-10T18:43:38.162991+0000 mon.b (mon.1) 262 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-55"}]: dispatch 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: audit 2024-09-10T18:43:38.177729+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: audit 2024-09-10T18:43:38.179182+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-55"}]: dispatch 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[18597]: cluster 2024-09-10T18:43:38.886001+0000 mon.a (mon.0) 2502 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: audit 2024-09-10T18:43:38.107113+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: audit 2024-09-10T18:43:38.117229+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: cluster 2024-09-10T18:43:38.117447+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:43:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: audit 2024-09-10T18:43:38.161422+0000 mon.b (mon.1) 261 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:40.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: audit 2024-09-10T18:43:38.162991+0000 mon.b (mon.1) 262 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-55"}]: dispatch 2024-09-10T18:43:40.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: audit 2024-09-10T18:43:38.177729+0000 mon.a (mon.0) 2500 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:40.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: audit 2024-09-10T18:43:38.179182+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-55"}]: dispatch 2024-09-10T18:43:40.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:39 smithi100 bash[31607]: cluster 2024-09-10T18:43:38.886001+0000 mon.a (mon.0) 2502 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:40.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:40 smithi151 bash[22565]: cluster 2024-09-10T18:43:39.125934+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:43:40.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:40 smithi151 bash[22565]: audit 2024-09-10T18:43:39.571418+0000 mon.a (mon.0) 2504 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:40.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:40 smithi151 bash[22565]: audit 2024-09-10T18:43:39.573135+0000 mon.c (mon.2) 437 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:40.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:40 smithi151 bash[22565]: cluster 2024-09-10T18:43:39.945987+0000 mgr.y (mgr.24512) 374 : cluster [DBG] pgmap v566: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:40.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:40 smithi151 bash[22565]: audit 2024-09-10T18:43:40.121491+0000 mon.b (mon.1) 263 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:40.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:40 smithi151 bash[22565]: cluster 2024-09-10T18:43:40.131637+0000 mon.a (mon.0) 2505 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:43:40.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:40 smithi151 bash[22565]: audit 2024-09-10T18:43:40.138777+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:41.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[18597]: cluster 2024-09-10T18:43:39.125934+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:43:41.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[18597]: audit 2024-09-10T18:43:39.571418+0000 mon.a (mon.0) 2504 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:41.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[18597]: audit 2024-09-10T18:43:39.573135+0000 mon.c (mon.2) 437 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[18597]: cluster 2024-09-10T18:43:39.945987+0000 mgr.y (mgr.24512) 374 : cluster [DBG] pgmap v566: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[18597]: audit 2024-09-10T18:43:40.121491+0000 mon.b (mon.1) 263 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[18597]: cluster 2024-09-10T18:43:40.131637+0000 mon.a (mon.0) 2505 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[18597]: audit 2024-09-10T18:43:40.138777+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[31607]: cluster 2024-09-10T18:43:39.125934+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[31607]: audit 2024-09-10T18:43:39.571418+0000 mon.a (mon.0) 2504 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[31607]: audit 2024-09-10T18:43:39.573135+0000 mon.c (mon.2) 437 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[31607]: cluster 2024-09-10T18:43:39.945987+0000 mgr.y (mgr.24512) 374 : cluster [DBG] pgmap v566: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[31607]: audit 2024-09-10T18:43:40.121491+0000 mon.b (mon.1) 263 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[31607]: cluster 2024-09-10T18:43:40.131637+0000 mon.a (mon.0) 2505 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:43:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:40 smithi100 bash[31607]: audit 2024-09-10T18:43:40.138777+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:43:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:41] "GET /metrics HTTP/1.1" 200 226138 "" "Prometheus/2.43.0" 2024-09-10T18:43:42.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:43:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[22565]: audit 2024-09-10T18:43:41.131297+0000 mon.a (mon.0) 2507 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[22565]: cluster 2024-09-10T18:43:41.131529+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:43:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[22565]: audit 2024-09-10T18:43:41.165070+0000 mon.b (mon.1) 264 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[22565]: audit 2024-09-10T18:43:41.166537+0000 mon.b (mon.1) 265 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-57"}]: dispatch 2024-09-10T18:43:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[22565]: audit 2024-09-10T18:43:41.181239+0000 mon.a (mon.0) 2509 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[22565]: audit 2024-09-10T18:43:41.182611+0000 mon.a (mon.0) 2510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-57"}]: dispatch 2024-09-10T18:43:42.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:42 smithi151 bash[22565]: cluster 2024-09-10T18:43:41.946844+0000 mgr.y (mgr.24512) 375 : cluster [DBG] pgmap v569: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:43.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[18597]: audit 2024-09-10T18:43:41.131297+0000 mon.a (mon.0) 2507 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:43.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[18597]: cluster 2024-09-10T18:43:41.131529+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:43:43.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[18597]: audit 2024-09-10T18:43:41.165070+0000 mon.b (mon.1) 264 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:43.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[18597]: audit 2024-09-10T18:43:41.166537+0000 mon.b (mon.1) 265 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-57"}]: dispatch 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[18597]: audit 2024-09-10T18:43:41.181239+0000 mon.a (mon.0) 2509 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[18597]: audit 2024-09-10T18:43:41.182611+0000 mon.a (mon.0) 2510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-57"}]: dispatch 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[18597]: cluster 2024-09-10T18:43:41.946844+0000 mgr.y (mgr.24512) 375 : cluster [DBG] pgmap v569: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[31607]: audit 2024-09-10T18:43:41.131297+0000 mon.a (mon.0) 2507 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[31607]: cluster 2024-09-10T18:43:41.131529+0000 mon.a (mon.0) 2508 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[31607]: audit 2024-09-10T18:43:41.165070+0000 mon.b (mon.1) 264 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[31607]: audit 2024-09-10T18:43:41.166537+0000 mon.b (mon.1) 265 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-57"}]: dispatch 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[31607]: audit 2024-09-10T18:43:41.181239+0000 mon.a (mon.0) 2509 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[31607]: audit 2024-09-10T18:43:41.182611+0000 mon.a (mon.0) 2510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-57"}]: dispatch 2024-09-10T18:43:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:42 smithi100 bash[31607]: cluster 2024-09-10T18:43:41.946844+0000 mgr.y (mgr.24512) 375 : cluster [DBG] pgmap v569: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:43.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:43 smithi151 bash[22565]: cluster 2024-09-10T18:43:42.138578+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:43:43.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:43 smithi151 bash[22565]: audit 2024-09-10T18:43:42.382706+0000 mgr.y (mgr.24512) 376 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:43.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:43 smithi151 bash[22565]: audit 2024-09-10T18:43:43.125691+0000 mon.b (mon.1) 266 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:43.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:43 smithi151 bash[22565]: cluster 2024-09-10T18:43:43.138044+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:43:43.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:43 smithi151 bash[22565]: audit 2024-09-10T18:43:43.142314+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:44.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[18597]: cluster 2024-09-10T18:43:42.138578+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:43:44.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[18597]: audit 2024-09-10T18:43:42.382706+0000 mgr.y (mgr.24512) 376 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:44.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[18597]: audit 2024-09-10T18:43:43.125691+0000 mon.b (mon.1) 266 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:44.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[18597]: cluster 2024-09-10T18:43:43.138044+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:43:44.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[18597]: audit 2024-09-10T18:43:43.142314+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[31607]: cluster 2024-09-10T18:43:42.138578+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:43:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[31607]: audit 2024-09-10T18:43:42.382706+0000 mgr.y (mgr.24512) 376 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[31607]: audit 2024-09-10T18:43:43.125691+0000 mon.b (mon.1) 266 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[31607]: cluster 2024-09-10T18:43:43.138044+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:43:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:43 smithi100 bash[31607]: audit 2024-09-10T18:43:43.142314+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:44.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: cluster 2024-09-10T18:43:43.947722+0000 mgr.y (mgr.24512) 377 : cluster [DBG] pgmap v572: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: audit 2024-09-10T18:43:44.129713+0000 mon.b (mon.1) 267 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: audit 2024-09-10T18:43:44.142155+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: cluster 2024-09-10T18:43:44.143123+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: audit 2024-09-10T18:43:44.180474+0000 mon.b (mon.1) 268 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: audit 2024-09-10T18:43:44.182287+0000 mon.b (mon.1) 269 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-59"}]: dispatch 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: audit 2024-09-10T18:43:44.196631+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:44.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:44 smithi151 bash[22565]: audit 2024-09-10T18:43:44.198291+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-59"}]: dispatch 2024-09-10T18:43:45.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: cluster 2024-09-10T18:43:43.947722+0000 mgr.y (mgr.24512) 377 : cluster [DBG] pgmap v572: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:45.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: audit 2024-09-10T18:43:44.129713+0000 mon.b (mon.1) 267 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: audit 2024-09-10T18:43:44.142155+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: cluster 2024-09-10T18:43:44.143123+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: audit 2024-09-10T18:43:44.180474+0000 mon.b (mon.1) 268 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: audit 2024-09-10T18:43:44.182287+0000 mon.b (mon.1) 269 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-59"}]: dispatch 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: audit 2024-09-10T18:43:44.196631+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[18597]: audit 2024-09-10T18:43:44.198291+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-59"}]: dispatch 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: cluster 2024-09-10T18:43:43.947722+0000 mgr.y (mgr.24512) 377 : cluster [DBG] pgmap v572: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: audit 2024-09-10T18:43:44.129713+0000 mon.b (mon.1) 267 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: audit 2024-09-10T18:43:44.142155+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: cluster 2024-09-10T18:43:44.143123+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:43:45.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: audit 2024-09-10T18:43:44.180474+0000 mon.b (mon.1) 268 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:45.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: audit 2024-09-10T18:43:44.182287+0000 mon.b (mon.1) 269 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-59"}]: dispatch 2024-09-10T18:43:45.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: audit 2024-09-10T18:43:44.196631+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:45.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:44 smithi100 bash[31607]: audit 2024-09-10T18:43:44.198291+0000 mon.a (mon.0) 2517 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-59"}]: dispatch 2024-09-10T18:43:45.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:45 smithi151 bash[22565]: cluster 2024-09-10T18:43:44.570356+0000 mon.a (mon.0) 2518 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:45.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:45 smithi151 bash[22565]: cluster 2024-09-10T18:43:45.146255+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:43:46.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:45 smithi100 bash[31607]: cluster 2024-09-10T18:43:44.570356+0000 mon.a (mon.0) 2518 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:46.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:45 smithi100 bash[31607]: cluster 2024-09-10T18:43:45.146255+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:43:46.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:45 smithi100 bash[18597]: cluster 2024-09-10T18:43:44.570356+0000 mon.a (mon.0) 2518 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:46.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:45 smithi100 bash[18597]: cluster 2024-09-10T18:43:45.146255+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:43:46.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:46 smithi151 bash[22565]: cluster 2024-09-10T18:43:45.949732+0000 mgr.y (mgr.24512) 378 : cluster [DBG] pgmap v575: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:43:46.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:46 smithi151 bash[22565]: audit 2024-09-10T18:43:46.139458+0000 mon.b (mon.1) 270 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:46.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:46 smithi151 bash[22565]: cluster 2024-09-10T18:43:46.148694+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:43:46.928 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:46 smithi151 bash[22565]: audit 2024-09-10T18:43:46.160229+0000 mon.a (mon.0) 2521 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:47.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[18597]: cluster 2024-09-10T18:43:45.949732+0000 mgr.y (mgr.24512) 378 : cluster [DBG] pgmap v575: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:43:47.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[18597]: audit 2024-09-10T18:43:46.139458+0000 mon.b (mon.1) 270 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:47.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[18597]: cluster 2024-09-10T18:43:46.148694+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:43:47.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[18597]: audit 2024-09-10T18:43:46.160229+0000 mon.a (mon.0) 2521 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:47.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[31607]: cluster 2024-09-10T18:43:45.949732+0000 mgr.y (mgr.24512) 378 : cluster [DBG] pgmap v575: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:43:47.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[31607]: audit 2024-09-10T18:43:46.139458+0000 mon.b (mon.1) 270 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:47.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[31607]: cluster 2024-09-10T18:43:46.148694+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:43:47.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:46 smithi100 bash[31607]: audit 2024-09-10T18:43:46.160229+0000 mon.a (mon.0) 2521 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:48.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:48 smithi151 bash[22565]: audit 2024-09-10T18:43:47.139519+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:48 smithi151 bash[22565]: audit 2024-09-10T18:43:47.143266+0000 mon.b (mon.1) 272 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:48 smithi151 bash[22565]: audit 2024-09-10T18:43:47.151928+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:48 smithi151 bash[22565]: cluster 2024-09-10T18:43:47.152521+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-09-10T18:43:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:48 smithi151 bash[22565]: audit 2024-09-10T18:43:47.159447+0000 mon.a (mon.0) 2524 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:48.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:48 smithi151 bash[22565]: cluster 2024-09-10T18:43:47.950540+0000 mgr.y (mgr.24512) 379 : cluster [DBG] pgmap v578: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:43:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[18597]: audit 2024-09-10T18:43:47.139519+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[18597]: audit 2024-09-10T18:43:47.143266+0000 mon.b (mon.1) 272 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[18597]: audit 2024-09-10T18:43:47.151928+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[18597]: cluster 2024-09-10T18:43:47.152521+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[18597]: audit 2024-09-10T18:43:47.159447+0000 mon.a (mon.0) 2524 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[18597]: cluster 2024-09-10T18:43:47.950540+0000 mgr.y (mgr.24512) 379 : cluster [DBG] pgmap v578: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[31607]: audit 2024-09-10T18:43:47.139519+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[31607]: audit 2024-09-10T18:43:47.143266+0000 mon.b (mon.1) 272 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[31607]: audit 2024-09-10T18:43:47.151928+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[31607]: cluster 2024-09-10T18:43:47.152521+0000 mon.a (mon.0) 2523 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[31607]: audit 2024-09-10T18:43:47.159447+0000 mon.a (mon.0) 2524 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:48 smithi100 bash[31607]: cluster 2024-09-10T18:43:47.950540+0000 mgr.y (mgr.24512) 379 : cluster [DBG] pgmap v578: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:43:49.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:49 smithi151 bash[22565]: audit 2024-09-10T18:43:48.153116+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:43:49.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:49 smithi151 bash[22565]: cluster 2024-09-10T18:43:48.153308+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-09-10T18:43:49.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:49 smithi151 bash[22565]: audit 2024-09-10T18:43:48.198305+0000 mon.b (mon.1) 273 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:49 smithi151 bash[22565]: audit 2024-09-10T18:43:48.200382+0000 mon.b (mon.1) 274 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-61"}]: dispatch 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:49 smithi151 bash[22565]: audit 2024-09-10T18:43:48.214518+0000 mon.a (mon.0) 2527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:49 smithi151 bash[22565]: audit 2024-09-10T18:43:48.216406+0000 mon.a (mon.0) 2528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-61"}]: dispatch 2024-09-10T18:43:49.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[18597]: audit 2024-09-10T18:43:48.153116+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:43:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[18597]: cluster 2024-09-10T18:43:48.153308+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-09-10T18:43:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[18597]: audit 2024-09-10T18:43:48.198305+0000 mon.b (mon.1) 273 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[18597]: audit 2024-09-10T18:43:48.200382+0000 mon.b (mon.1) 274 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-61"}]: dispatch 2024-09-10T18:43:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[18597]: audit 2024-09-10T18:43:48.214518+0000 mon.a (mon.0) 2527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[18597]: audit 2024-09-10T18:43:48.216406+0000 mon.a (mon.0) 2528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-61"}]: dispatch 2024-09-10T18:43:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[31607]: audit 2024-09-10T18:43:48.153116+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:43:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[31607]: cluster 2024-09-10T18:43:48.153308+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-09-10T18:43:49.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[31607]: audit 2024-09-10T18:43:48.198305+0000 mon.b (mon.1) 273 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:49.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[31607]: audit 2024-09-10T18:43:48.200382+0000 mon.b (mon.1) 274 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-61"}]: dispatch 2024-09-10T18:43:49.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[31607]: audit 2024-09-10T18:43:48.214518+0000 mon.a (mon.0) 2527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:43:49.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:49 smithi100 bash[31607]: audit 2024-09-10T18:43:48.216406+0000 mon.a (mon.0) 2528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-61"}]: dispatch 2024-09-10T18:43:50.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:50 smithi100 bash[18597]: cluster 2024-09-10T18:43:49.180847+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-09-10T18:43:50.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:50 smithi100 bash[18597]: cluster 2024-09-10T18:43:49.951304+0000 mgr.y (mgr.24512) 380 : cluster [DBG] pgmap v581: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:50.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:50 smithi100 bash[31607]: cluster 2024-09-10T18:43:49.180847+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-09-10T18:43:50.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:50 smithi100 bash[31607]: cluster 2024-09-10T18:43:49.951304+0000 mgr.y (mgr.24512) 380 : cluster [DBG] pgmap v581: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:50.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:50 smithi151 bash[22565]: cluster 2024-09-10T18:43:49.180847+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-09-10T18:43:50.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:50 smithi151 bash[22565]: cluster 2024-09-10T18:43:49.951304+0000 mgr.y (mgr.24512) 380 : cluster [DBG] pgmap v581: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:51.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:51] "GET /metrics HTTP/1.1" 200 226138 "" "Prometheus/2.43.0" 2024-09-10T18:43:51.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[31607]: audit 2024-09-10T18:43:50.181859+0000 mon.b (mon.1) 275 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[31607]: cluster 2024-09-10T18:43:50.192165+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-09-10T18:43:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[31607]: audit 2024-09-10T18:43:50.197716+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[31607]: cluster 2024-09-10T18:43:50.577782+0000 mon.a (mon.0) 2532 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[18597]: audit 2024-09-10T18:43:50.181859+0000 mon.b (mon.1) 275 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[18597]: cluster 2024-09-10T18:43:50.192165+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-09-10T18:43:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[18597]: audit 2024-09-10T18:43:50.197716+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:51 smithi100 bash[18597]: cluster 2024-09-10T18:43:50.577782+0000 mon.a (mon.0) 2532 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:51.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:51 smithi151 bash[22565]: audit 2024-09-10T18:43:50.181859+0000 mon.b (mon.1) 275 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:51.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:51 smithi151 bash[22565]: cluster 2024-09-10T18:43:50.192165+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-09-10T18:43:51.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:51 smithi151 bash[22565]: audit 2024-09-10T18:43:50.197716+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:51.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:51 smithi151 bash[22565]: cluster 2024-09-10T18:43:50.577782+0000 mon.a (mon.0) 2532 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[18597]: audit 2024-09-10T18:43:51.195093+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[18597]: audit 2024-09-10T18:43:51.197738+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[18597]: cluster 2024-09-10T18:43:51.197873+0000 mon.a (mon.0) 2534 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[18597]: audit 2024-09-10T18:43:51.198452+0000 mon.b (mon.1) 277 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[18597]: audit 2024-09-10T18:43:51.214407+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[18597]: cluster 2024-09-10T18:43:51.952111+0000 mgr.y (mgr.24512) 381 : cluster [DBG] pgmap v584: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[31607]: audit 2024-09-10T18:43:51.195093+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[31607]: audit 2024-09-10T18:43:51.197738+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[31607]: cluster 2024-09-10T18:43:51.197873+0000 mon.a (mon.0) 2534 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[31607]: audit 2024-09-10T18:43:51.198452+0000 mon.b (mon.1) 277 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[31607]: audit 2024-09-10T18:43:51.214407+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:52.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:52 smithi100 bash[31607]: cluster 2024-09-10T18:43:51.952111+0000 mgr.y (mgr.24512) 381 : cluster [DBG] pgmap v584: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:52.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:43:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:43:52.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:52 smithi151 bash[22565]: audit 2024-09-10T18:43:51.195093+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:43:52.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:52 smithi151 bash[22565]: audit 2024-09-10T18:43:51.197738+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:52.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:52 smithi151 bash[22565]: cluster 2024-09-10T18:43:51.197873+0000 mon.a (mon.0) 2534 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:52 smithi151 bash[22565]: audit 2024-09-10T18:43:51.198452+0000 mon.b (mon.1) 277 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:52 smithi151 bash[22565]: audit 2024-09-10T18:43:51.214407+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:52 smithi151 bash[22565]: cluster 2024-09-10T18:43:51.952111+0000 mgr.y (mgr.24512) 381 : cluster [DBG] pgmap v584: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:53.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:53 smithi100 bash[18597]: audit 2024-09-10T18:43:52.210937+0000 mon.a (mon.0) 2536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:43:53.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:53 smithi100 bash[18597]: cluster 2024-09-10T18:43:52.211042+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-09-10T18:43:53.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:53 smithi100 bash[18597]: audit 2024-09-10T18:43:52.393784+0000 mgr.y (mgr.24512) 382 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:53.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:53 smithi100 bash[31607]: audit 2024-09-10T18:43:52.210937+0000 mon.a (mon.0) 2536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:43:53.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:53 smithi100 bash[31607]: cluster 2024-09-10T18:43:52.211042+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-09-10T18:43:53.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:53 smithi100 bash[31607]: audit 2024-09-10T18:43:52.393784+0000 mgr.y (mgr.24512) 382 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:53.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:53 smithi151 bash[22565]: audit 2024-09-10T18:43:52.210937+0000 mon.a (mon.0) 2536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:43:53.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:53 smithi151 bash[22565]: cluster 2024-09-10T18:43:52.211042+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-09-10T18:43:53.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:53 smithi151 bash[22565]: audit 2024-09-10T18:43:52.393784+0000 mgr.y (mgr.24512) 382 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:54 smithi100 bash[18597]: cluster 2024-09-10T18:43:53.231888+0000 mon.a (mon.0) 2538 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-09-10T18:43:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:54 smithi100 bash[18597]: cluster 2024-09-10T18:43:53.952981+0000 mgr.y (mgr.24512) 383 : cluster [DBG] pgmap v587: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:54.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:54 smithi100 bash[31607]: cluster 2024-09-10T18:43:53.231888+0000 mon.a (mon.0) 2538 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-09-10T18:43:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:54 smithi100 bash[31607]: cluster 2024-09-10T18:43:53.952981+0000 mgr.y (mgr.24512) 383 : cluster [DBG] pgmap v587: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:54.553 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:54 smithi151 bash[22565]: cluster 2024-09-10T18:43:53.231888+0000 mon.a (mon.0) 2538 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-09-10T18:43:54.553 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:54 smithi151 bash[22565]: cluster 2024-09-10T18:43:53.952981+0000 mgr.y (mgr.24512) 383 : cluster [DBG] pgmap v587: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:43:54.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:43:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:43:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:55.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:55 smithi100 bash[18597]: cluster 2024-09-10T18:43:54.241752+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-09-10T18:43:55.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:55 smithi100 bash[18597]: audit 2024-09-10T18:43:54.330492+0000 mon.c (mon.2) 438 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:55.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:55 smithi100 bash[31607]: cluster 2024-09-10T18:43:54.241752+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-09-10T18:43:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:55 smithi100 bash[31607]: audit 2024-09-10T18:43:54.330492+0000 mon.c (mon.2) 438 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:55.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:55 smithi151 bash[22565]: cluster 2024-09-10T18:43:54.241752+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-09-10T18:43:55.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:55 smithi151 bash[22565]: audit 2024-09-10T18:43:54.330492+0000 mon.c (mon.2) 438 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:56.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:56 smithi100 bash[18597]: cluster 2024-09-10T18:43:55.246632+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-09-10T18:43:56.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:56 smithi100 bash[18597]: cluster 2024-09-10T18:43:55.955225+0000 mgr.y (mgr.24512) 384 : cluster [DBG] pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T18:43:56.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:56 smithi100 bash[31607]: cluster 2024-09-10T18:43:55.246632+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-09-10T18:43:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:56 smithi100 bash[31607]: cluster 2024-09-10T18:43:55.955225+0000 mgr.y (mgr.24512) 384 : cluster [DBG] pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T18:43:56.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:56 smithi151 bash[22565]: cluster 2024-09-10T18:43:55.246632+0000 mon.a (mon.0) 2540 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-09-10T18:43:56.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:56 smithi151 bash[22565]: cluster 2024-09-10T18:43:55.955225+0000 mgr.y (mgr.24512) 384 : cluster [DBG] pgmap v590: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T18:43:57.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:57 smithi100 bash[18597]: cluster 2024-09-10T18:43:56.264009+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-09-10T18:43:57.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:57 smithi100 bash[31607]: cluster 2024-09-10T18:43:56.264009+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-09-10T18:43:57.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:57 smithi151 bash[22565]: cluster 2024-09-10T18:43:56.264009+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-09-10T18:43:58.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:43:58 smithi100 bash[18597]: cluster 2024-09-10T18:43:57.956169+0000 mgr.y (mgr.24512) 385 : cluster [DBG] pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-09-10T18:43:58.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:43:58 smithi100 bash[31607]: cluster 2024-09-10T18:43:57.956169+0000 mgr.y (mgr.24512) 385 : cluster [DBG] pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-09-10T18:43:58.677 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:43:58 smithi151 bash[22565]: cluster 2024-09-10T18:43:57.956169+0000 mgr.y (mgr.24512) 385 : cluster [DBG] pgmap v592: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-09-10T18:44:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:01 smithi100 bash[18597]: cluster 2024-09-10T18:43:59.957112+0000 mgr.y (mgr.24512) 386 : cluster [DBG] pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-10T18:44:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:01 smithi100 bash[31607]: cluster 2024-09-10T18:43:59.957112+0000 mgr.y (mgr.24512) 386 : cluster [DBG] pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:01 smithi151 bash[22565]: cluster 2024-09-10T18:43:59.957112+0000 mgr.y (mgr.24512) 386 : cluster [DBG] pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-10T18:44:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:44:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:01] "GET /metrics HTTP/1.1" 200 228584 "" "Prometheus/2.43.0" 2024-09-10T18:44:02.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:44:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:44:03.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:03 smithi100 bash[18597]: cluster 2024-09-10T18:44:01.959348+0000 mgr.y (mgr.24512) 387 : cluster [DBG] pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-10T18:44:03.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:03 smithi100 bash[31607]: cluster 2024-09-10T18:44:01.959348+0000 mgr.y (mgr.24512) 387 : cluster [DBG] pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:03 smithi151 bash[22565]: cluster 2024-09-10T18:44:01.959348+0000 mgr.y (mgr.24512) 387 : cluster [DBG] pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-10T18:44:04.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:04 smithi100 bash[18597]: audit 2024-09-10T18:44:02.404993+0000 mgr.y (mgr.24512) 388 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:04.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:04 smithi100 bash[31607]: audit 2024-09-10T18:44:02.404993+0000 mgr.y (mgr.24512) 388 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:04.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:04 smithi151 bash[22565]: audit 2024-09-10T18:44:02.404993+0000 mgr.y (mgr.24512) 388 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:04.927 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:44:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:05.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:05 smithi100 bash[18597]: cluster 2024-09-10T18:44:03.960162+0000 mgr.y (mgr.24512) 389 : cluster [DBG] pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-10T18:44:05.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:05 smithi100 bash[31607]: cluster 2024-09-10T18:44:03.960162+0000 mgr.y (mgr.24512) 389 : cluster [DBG] pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-10T18:44:05.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:05 smithi151 bash[22565]: cluster 2024-09-10T18:44:03.960162+0000 mgr.y (mgr.24512) 389 : cluster [DBG] pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-10T18:44:06.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:06 smithi151 bash[22565]: cluster 2024-09-10T18:44:05.591063+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-09-10T18:44:06.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:06 smithi151 bash[22565]: cluster 2024-09-10T18:44:05.960995+0000 mgr.y (mgr.24512) 390 : cluster [DBG] pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:07.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:06 smithi100 bash[18597]: cluster 2024-09-10T18:44:05.591063+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-09-10T18:44:07.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:06 smithi100 bash[18597]: cluster 2024-09-10T18:44:05.960995+0000 mgr.y (mgr.24512) 390 : cluster [DBG] pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:07.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:06 smithi100 bash[31607]: cluster 2024-09-10T18:44:05.591063+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-09-10T18:44:07.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:06 smithi100 bash[31607]: cluster 2024-09-10T18:44:05.960995+0000 mgr.y (mgr.24512) 390 : cluster [DBG] pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:07.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:07 smithi151 bash[22565]: cluster 2024-09-10T18:44:06.605847+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-09-10T18:44:08.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:07 smithi100 bash[18597]: cluster 2024-09-10T18:44:06.605847+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-09-10T18:44:08.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:07 smithi100 bash[31607]: cluster 2024-09-10T18:44:06.605847+0000 mon.a (mon.0) 2543 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-09-10T18:44:08.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:08 smithi151 bash[22565]: cluster 2024-09-10T18:44:07.961866+0000 mgr.y (mgr.24512) 391 : cluster [DBG] pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:09.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:08 smithi100 bash[18597]: cluster 2024-09-10T18:44:07.961866+0000 mgr.y (mgr.24512) 391 : cluster [DBG] pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:09.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:08 smithi100 bash[31607]: cluster 2024-09-10T18:44:07.961866+0000 mgr.y (mgr.24512) 391 : cluster [DBG] pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:09.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:09 smithi151 bash[22565]: audit 2024-09-10T18:44:09.331044+0000 mon.c (mon.2) 439 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:10.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:09 smithi100 bash[18597]: audit 2024-09-10T18:44:09.331044+0000 mon.c (mon.2) 439 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:10.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:09 smithi100 bash[31607]: audit 2024-09-10T18:44:09.331044+0000 mon.c (mon.2) 439 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:10.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:10 smithi151 bash[22565]: cluster 2024-09-10T18:44:09.962692+0000 mgr.y (mgr.24512) 392 : cluster [DBG] pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:44:11.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:10 smithi100 bash[18597]: cluster 2024-09-10T18:44:09.962692+0000 mgr.y (mgr.24512) 392 : cluster [DBG] pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:44:11.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:10 smithi100 bash[31607]: cluster 2024-09-10T18:44:09.962692+0000 mgr.y (mgr.24512) 392 : cluster [DBG] pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:44:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:44:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:11] "GET /metrics HTTP/1.1" 200 228564 "" "Prometheus/2.43.0" 2024-09-10T18:44:12.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:44:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:44:13.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:13 smithi100 bash[31607]: cluster 2024-09-10T18:44:11.964924+0000 mgr.y (mgr.24512) 393 : cluster [DBG] pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:13.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:13 smithi100 bash[18597]: cluster 2024-09-10T18:44:11.964924+0000 mgr.y (mgr.24512) 393 : cluster [DBG] pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:13.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:13 smithi151 bash[22565]: cluster 2024-09-10T18:44:11.964924+0000 mgr.y (mgr.24512) 393 : cluster [DBG] pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:14.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:14 smithi100 bash[31607]: audit 2024-09-10T18:44:12.413750+0000 mgr.y (mgr.24512) 394 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:14.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:14 smithi100 bash[18597]: audit 2024-09-10T18:44:12.413750+0000 mgr.y (mgr.24512) 394 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:14.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:14 smithi151 bash[22565]: audit 2024-09-10T18:44:12.413750+0000 mgr.y (mgr.24512) 394 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:14.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:44:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:15.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:15 smithi100 bash[31607]: cluster 2024-09-10T18:44:13.965803+0000 mgr.y (mgr.24512) 395 : cluster [DBG] pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:15.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:15 smithi100 bash[18597]: cluster 2024-09-10T18:44:13.965803+0000 mgr.y (mgr.24512) 395 : cluster [DBG] pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:15.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:15 smithi151 bash[22565]: cluster 2024-09-10T18:44:13.965803+0000 mgr.y (mgr.24512) 395 : cluster [DBG] pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 977 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:16.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:16 smithi151 bash[22565]: cluster 2024-09-10T18:44:15.595912+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-09-10T18:44:16.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:16 smithi151 bash[22565]: cluster 2024-09-10T18:44:15.966628+0000 mgr.y (mgr.24512) 396 : cluster [DBG] pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:16 smithi100 bash[18597]: cluster 2024-09-10T18:44:15.595912+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-09-10T18:44:17.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:16 smithi100 bash[18597]: cluster 2024-09-10T18:44:15.966628+0000 mgr.y (mgr.24512) 396 : cluster [DBG] pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:16 smithi100 bash[31607]: cluster 2024-09-10T18:44:15.595912+0000 mon.a (mon.0) 2544 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-09-10T18:44:17.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:16 smithi100 bash[31607]: cluster 2024-09-10T18:44:15.966628+0000 mgr.y (mgr.24512) 396 : cluster [DBG] pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:18.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:18 smithi151 bash[22565]: cluster 2024-09-10T18:44:17.608716+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-09-10T18:44:18.927 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:18 smithi151 bash[22565]: cluster 2024-09-10T18:44:17.967277+0000 mgr.y (mgr.24512) 397 : cluster [DBG] pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:18 smithi100 bash[18597]: cluster 2024-09-10T18:44:17.608716+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-09-10T18:44:19.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:18 smithi100 bash[18597]: cluster 2024-09-10T18:44:17.967277+0000 mgr.y (mgr.24512) 397 : cluster [DBG] pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:18 smithi100 bash[31607]: cluster 2024-09-10T18:44:17.608716+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-09-10T18:44:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:18 smithi100 bash[31607]: cluster 2024-09-10T18:44:17.967277+0000 mgr.y (mgr.24512) 397 : cluster [DBG] pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:44:21.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:21 smithi100 bash[18597]: cluster 2024-09-10T18:44:19.968051+0000 mgr.y (mgr.24512) 398 : cluster [DBG] pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:44:21.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:21 smithi100 bash[31607]: cluster 2024-09-10T18:44:19.968051+0000 mgr.y (mgr.24512) 398 : cluster [DBG] pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:44:21.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:21 smithi151 bash[22565]: cluster 2024-09-10T18:44:19.968051+0000 mgr.y (mgr.24512) 398 : cluster [DBG] pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:44:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:44:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:21] "GET /metrics HTTP/1.1" 200 228564 "" "Prometheus/2.43.0" 2024-09-10T18:44:22.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:44:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:44:23.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:23 smithi100 bash[18597]: cluster 2024-09-10T18:44:21.970310+0000 mgr.y (mgr.24512) 399 : cluster [DBG] pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-09-10T18:44:23.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:23 smithi100 bash[31607]: cluster 2024-09-10T18:44:21.970310+0000 mgr.y (mgr.24512) 399 : cluster [DBG] pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-09-10T18:44:23.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:23 smithi151 bash[22565]: cluster 2024-09-10T18:44:21.970310+0000 mgr.y (mgr.24512) 399 : cluster [DBG] pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-09-10T18:44:24.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:24 smithi100 bash[18597]: audit 2024-09-10T18:44:22.424858+0000 mgr.y (mgr.24512) 400 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:24.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:24 smithi100 bash[31607]: audit 2024-09-10T18:44:22.424858+0000 mgr.y (mgr.24512) 400 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:24.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:24 smithi151 bash[22565]: audit 2024-09-10T18:44:22.424858+0000 mgr.y (mgr.24512) 400 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:24.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:44:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:25.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:25 smithi100 bash[31607]: cluster 2024-09-10T18:44:23.971133+0000 mgr.y (mgr.24512) 401 : cluster [DBG] pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 52 KiB/s rd, 0 B/s wr, 86 op/s 2024-09-10T18:44:25.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:25 smithi100 bash[31607]: audit 2024-09-10T18:44:24.331485+0000 mon.c (mon.2) 440 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:25 smithi100 bash[18597]: cluster 2024-09-10T18:44:23.971133+0000 mgr.y (mgr.24512) 401 : cluster [DBG] pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 52 KiB/s rd, 0 B/s wr, 86 op/s 2024-09-10T18:44:25.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:25 smithi100 bash[18597]: audit 2024-09-10T18:44:24.331485+0000 mon.c (mon.2) 440 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:25.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:25 smithi151 bash[22565]: cluster 2024-09-10T18:44:23.971133+0000 mgr.y (mgr.24512) 401 : cluster [DBG] pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 52 KiB/s rd, 0 B/s wr, 86 op/s 2024-09-10T18:44:25.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:25 smithi151 bash[22565]: audit 2024-09-10T18:44:24.331485+0000 mon.c (mon.2) 440 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:26.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:26 smithi151 bash[22565]: cluster 2024-09-10T18:44:25.597281+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-09-10T18:44:26.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:26 smithi151 bash[22565]: cluster 2024-09-10T18:44:25.972022+0000 mgr.y (mgr.24512) 402 : cluster [DBG] pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 86 op/s 2024-09-10T18:44:27.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:26 smithi100 bash[31607]: cluster 2024-09-10T18:44:25.597281+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-09-10T18:44:27.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:26 smithi100 bash[31607]: cluster 2024-09-10T18:44:25.972022+0000 mgr.y (mgr.24512) 402 : cluster [DBG] pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 86 op/s 2024-09-10T18:44:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:26 smithi100 bash[18597]: cluster 2024-09-10T18:44:25.597281+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-09-10T18:44:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:26 smithi100 bash[18597]: cluster 2024-09-10T18:44:25.972022+0000 mgr.y (mgr.24512) 402 : cluster [DBG] pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 86 op/s 2024-09-10T18:44:28.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[31607]: audit 2024-09-10T18:44:27.631948+0000 mon.b (mon.1) 278 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[31607]: audit 2024-09-10T18:44:27.634324+0000 mon.b (mon.1) 279 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-63"}]: dispatch 2024-09-10T18:44:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[31607]: audit 2024-09-10T18:44:27.646961+0000 mon.a (mon.0) 2547 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[31607]: audit 2024-09-10T18:44:27.649280+0000 mon.a (mon.0) 2548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-63"}]: dispatch 2024-09-10T18:44:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[18597]: audit 2024-09-10T18:44:27.631948+0000 mon.b (mon.1) 278 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:28.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[18597]: audit 2024-09-10T18:44:27.634324+0000 mon.b (mon.1) 279 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-63"}]: dispatch 2024-09-10T18:44:28.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[18597]: audit 2024-09-10T18:44:27.646961+0000 mon.a (mon.0) 2547 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:28.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:27 smithi100 bash[18597]: audit 2024-09-10T18:44:27.649280+0000 mon.a (mon.0) 2548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-63"}]: dispatch 2024-09-10T18:44:28.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:27 smithi151 bash[22565]: audit 2024-09-10T18:44:27.631948+0000 mon.b (mon.1) 278 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:28.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:27 smithi151 bash[22565]: audit 2024-09-10T18:44:27.634324+0000 mon.b (mon.1) 279 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-63"}]: dispatch 2024-09-10T18:44:28.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:27 smithi151 bash[22565]: audit 2024-09-10T18:44:27.646961+0000 mon.a (mon.0) 2547 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:28.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:27 smithi151 bash[22565]: audit 2024-09-10T18:44:27.649280+0000 mon.a (mon.0) 2548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-63"}]: dispatch 2024-09-10T18:44:29.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:28 smithi100 bash[18597]: cluster 2024-09-10T18:44:27.972906+0000 mgr.y (mgr.24512) 403 : cluster [DBG] pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-09-10T18:44:29.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:28 smithi100 bash[31607]: cluster 2024-09-10T18:44:27.972906+0000 mgr.y (mgr.24512) 403 : cluster [DBG] pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-09-10T18:44:29.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:28 smithi151 bash[22565]: cluster 2024-09-10T18:44:27.972906+0000 mgr.y (mgr.24512) 403 : cluster [DBG] pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-09-10T18:44:30.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:29 smithi100 bash[31607]: cluster 2024-09-10T18:44:28.709202+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-09-10T18:44:30.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:29 smithi100 bash[18597]: cluster 2024-09-10T18:44:28.709202+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-09-10T18:44:30.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:29 smithi151 bash[22565]: cluster 2024-09-10T18:44:28.709202+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-09-10T18:44:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[31607]: audit 2024-09-10T18:44:29.723567+0000 mon.b (mon.1) 280 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[31607]: cluster 2024-09-10T18:44:29.732766+0000 mon.a (mon.0) 2550 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-09-10T18:44:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[31607]: audit 2024-09-10T18:44:29.740507+0000 mon.a (mon.0) 2551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[31607]: cluster 2024-09-10T18:44:29.973787+0000 mgr.y (mgr.24512) 404 : cluster [DBG] pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[18597]: audit 2024-09-10T18:44:29.723567+0000 mon.b (mon.1) 280 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[18597]: cluster 2024-09-10T18:44:29.732766+0000 mon.a (mon.0) 2550 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-09-10T18:44:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[18597]: audit 2024-09-10T18:44:29.740507+0000 mon.a (mon.0) 2551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:30 smithi100 bash[18597]: cluster 2024-09-10T18:44:29.973787+0000 mgr.y (mgr.24512) 404 : cluster [DBG] pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:31.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:30 smithi151 bash[22565]: audit 2024-09-10T18:44:29.723567+0000 mon.b (mon.1) 280 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:30 smithi151 bash[22565]: cluster 2024-09-10T18:44:29.732766+0000 mon.a (mon.0) 2550 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-09-10T18:44:31.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:30 smithi151 bash[22565]: audit 2024-09-10T18:44:29.740507+0000 mon.a (mon.0) 2551 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:30 smithi151 bash[22565]: cluster 2024-09-10T18:44:29.973787+0000 mgr.y (mgr.24512) 404 : cluster [DBG] pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:31.743 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:31] "GET /metrics HTTP/1.1" 200 228600 "" "Prometheus/2.43.0" 2024-09-10T18:44:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[18597]: audit 2024-09-10T18:44:30.725672+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[18597]: cluster 2024-09-10T18:44:30.727451+0000 mon.a (mon.0) 2552 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[18597]: audit 2024-09-10T18:44:30.736628+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:32.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[18597]: audit 2024-09-10T18:44:30.736729+0000 mon.b (mon.1) 282 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[18597]: cluster 2024-09-10T18:44:30.736842+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[18597]: audit 2024-09-10T18:44:30.751548+0000 mon.a (mon.0) 2555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[31607]: audit 2024-09-10T18:44:30.725672+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[31607]: cluster 2024-09-10T18:44:30.727451+0000 mon.a (mon.0) 2552 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[31607]: audit 2024-09-10T18:44:30.736628+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[31607]: audit 2024-09-10T18:44:30.736729+0000 mon.b (mon.1) 282 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[31607]: cluster 2024-09-10T18:44:30.736842+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-09-10T18:44:32.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:31 smithi100 bash[31607]: audit 2024-09-10T18:44:30.751548+0000 mon.a (mon.0) 2555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:32.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:31 smithi151 bash[22565]: audit 2024-09-10T18:44:30.725672+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:32.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:31 smithi151 bash[22565]: cluster 2024-09-10T18:44:30.727451+0000 mon.a (mon.0) 2552 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:32.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:31 smithi151 bash[22565]: audit 2024-09-10T18:44:30.736628+0000 mon.a (mon.0) 2553 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:32.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:31 smithi151 bash[22565]: audit 2024-09-10T18:44:30.736729+0000 mon.b (mon.1) 282 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:32.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:31 smithi151 bash[22565]: cluster 2024-09-10T18:44:30.736842+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-09-10T18:44:32.177 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:31 smithi151 bash[22565]: audit 2024-09-10T18:44:30.751548+0000 mon.a (mon.0) 2555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:32.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:44:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:44:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:32 smithi100 bash[31607]: audit 2024-09-10T18:44:31.756297+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:44:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:32 smithi100 bash[31607]: cluster 2024-09-10T18:44:31.756442+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-09-10T18:44:33.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:32 smithi100 bash[31607]: cluster 2024-09-10T18:44:31.974634+0000 mgr.y (mgr.24512) 405 : cluster [DBG] pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:32 smithi100 bash[18597]: audit 2024-09-10T18:44:31.756297+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:44:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:32 smithi100 bash[18597]: cluster 2024-09-10T18:44:31.756442+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-09-10T18:44:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:32 smithi100 bash[18597]: cluster 2024-09-10T18:44:31.974634+0000 mgr.y (mgr.24512) 405 : cluster [DBG] pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:33.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:32 smithi151 bash[22565]: audit 2024-09-10T18:44:31.756297+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:44:33.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:32 smithi151 bash[22565]: cluster 2024-09-10T18:44:31.756442+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-09-10T18:44:33.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:32 smithi151 bash[22565]: cluster 2024-09-10T18:44:31.974634+0000 mgr.y (mgr.24512) 405 : cluster [DBG] pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:34.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:33 smithi100 bash[31607]: audit 2024-09-10T18:44:32.435961+0000 mgr.y (mgr.24512) 406 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:34.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:33 smithi100 bash[31607]: cluster 2024-09-10T18:44:32.769258+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-09-10T18:44:34.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:33 smithi100 bash[18597]: audit 2024-09-10T18:44:32.435961+0000 mgr.y (mgr.24512) 406 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:34.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:33 smithi100 bash[18597]: cluster 2024-09-10T18:44:32.769258+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-09-10T18:44:34.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:33 smithi151 bash[22565]: audit 2024-09-10T18:44:32.435961+0000 mgr.y (mgr.24512) 406 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:34.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:33 smithi151 bash[22565]: cluster 2024-09-10T18:44:32.769258+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-09-10T18:44:34.926 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:44:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:34.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:34 smithi151 bash[22565]: cluster 2024-09-10T18:44:33.774947+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-09-10T18:44:34.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:34 smithi151 bash[22565]: cluster 2024-09-10T18:44:33.975455+0000 mgr.y (mgr.24512) 407 : cluster [DBG] pgmap v621: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:35.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:34 smithi100 bash[31607]: cluster 2024-09-10T18:44:33.774947+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-09-10T18:44:35.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:34 smithi100 bash[31607]: cluster 2024-09-10T18:44:33.975455+0000 mgr.y (mgr.24512) 407 : cluster [DBG] pgmap v621: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:34 smithi100 bash[18597]: cluster 2024-09-10T18:44:33.774947+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-09-10T18:44:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:34 smithi100 bash[18597]: cluster 2024-09-10T18:44:33.975455+0000 mgr.y (mgr.24512) 407 : cluster [DBG] pgmap v621: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:36.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:35 smithi100 bash[18597]: cluster 2024-09-10T18:44:34.794926+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-09-10T18:44:36.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:35 smithi100 bash[31607]: cluster 2024-09-10T18:44:34.794926+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-09-10T18:44:36.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:35 smithi151 bash[22565]: cluster 2024-09-10T18:44:34.794926+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-09-10T18:44:37.176 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:36 smithi151 bash[22565]: cluster 2024-09-10T18:44:35.977819+0000 mgr.y (mgr.24512) 408 : cluster [DBG] pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-10T18:44:37.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:36 smithi100 bash[18597]: cluster 2024-09-10T18:44:35.977819+0000 mgr.y (mgr.24512) 408 : cluster [DBG] pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-10T18:44:37.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:36 smithi100 bash[31607]: cluster 2024-09-10T18:44:35.977819+0000 mgr.y (mgr.24512) 408 : cluster [DBG] pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-10T18:44:38.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:37 smithi100 bash[31607]: audit 2024-09-10T18:44:37.938645+0000 mon.c (mon.2) 441 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:37 smithi100 bash[18597]: audit 2024-09-10T18:44:37.938645+0000 mon.c (mon.2) 441 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:38.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:37 smithi151 bash[22565]: audit 2024-09-10T18:44:37.938645+0000 mon.c (mon.2) 441 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[31607]: cluster 2024-09-10T18:44:37.978756+0000 mgr.y (mgr.24512) 409 : cluster [DBG] pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T18:44:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[31607]: audit 2024-09-10T18:44:38.248404+0000 mon.a (mon.0) 2561 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[31607]: audit 2024-09-10T18:44:38.260494+0000 mon.a (mon.0) 2562 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[31607]: audit 2024-09-10T18:44:38.696419+0000 mon.c (mon.2) 442 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[31607]: audit 2024-09-10T18:44:38.698016+0000 mon.c (mon.2) 443 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:39.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[31607]: audit 2024-09-10T18:44:38.708056+0000 mon.a (mon.0) 2563 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[18597]: cluster 2024-09-10T18:44:37.978756+0000 mgr.y (mgr.24512) 409 : cluster [DBG] pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T18:44:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[18597]: audit 2024-09-10T18:44:38.248404+0000 mon.a (mon.0) 2561 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[18597]: audit 2024-09-10T18:44:38.260494+0000 mon.a (mon.0) 2562 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[18597]: audit 2024-09-10T18:44:38.696419+0000 mon.c (mon.2) 442 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:39.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[18597]: audit 2024-09-10T18:44:38.698016+0000 mon.c (mon.2) 443 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:39.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:39 smithi100 bash[18597]: audit 2024-09-10T18:44:38.708056+0000 mon.a (mon.0) 2563 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:39 smithi151 bash[22565]: cluster 2024-09-10T18:44:37.978756+0000 mgr.y (mgr.24512) 409 : cluster [DBG] pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T18:44:39.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:39 smithi151 bash[22565]: audit 2024-09-10T18:44:38.248404+0000 mon.a (mon.0) 2561 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:39 smithi151 bash[22565]: audit 2024-09-10T18:44:38.260494+0000 mon.a (mon.0) 2562 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:39.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:39 smithi151 bash[22565]: audit 2024-09-10T18:44:38.696419+0000 mon.c (mon.2) 442 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:39.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:39 smithi151 bash[22565]: audit 2024-09-10T18:44:38.698016+0000 mon.c (mon.2) 443 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:39.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:39 smithi151 bash[22565]: audit 2024-09-10T18:44:38.708056+0000 mon.a (mon.0) 2563 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:44:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:40 smithi151 bash[22565]: audit 2024-09-10T18:44:39.331876+0000 mon.c (mon.2) 444 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:40.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:40 smithi100 bash[18597]: audit 2024-09-10T18:44:39.331876+0000 mon.c (mon.2) 444 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:40.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:40 smithi100 bash[31607]: audit 2024-09-10T18:44:39.331876+0000 mon.c (mon.2) 444 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:41.345 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:41 smithi100 bash[18597]: cluster 2024-09-10T18:44:39.979602+0000 mgr.y (mgr.24512) 410 : cluster [DBG] pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T18:44:41.346 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:41 smithi100 bash[18597]: cluster 2024-09-10T18:44:40.593182+0000 mon.a (mon.0) 2564 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:41.346 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:41 smithi100 bash[18597]: cluster 2024-09-10T18:44:40.608581+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-09-10T18:44:41.346 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:41 smithi100 bash[31607]: cluster 2024-09-10T18:44:39.979602+0000 mgr.y (mgr.24512) 410 : cluster [DBG] pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T18:44:41.346 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:41 smithi100 bash[31607]: cluster 2024-09-10T18:44:40.593182+0000 mon.a (mon.0) 2564 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:41.346 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:41 smithi100 bash[31607]: cluster 2024-09-10T18:44:40.608581+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-09-10T18:44:41.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:41 smithi151 bash[22565]: cluster 2024-09-10T18:44:39.979602+0000 mgr.y (mgr.24512) 410 : cluster [DBG] pgmap v625: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T18:44:41.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:41 smithi151 bash[22565]: cluster 2024-09-10T18:44:40.593182+0000 mon.a (mon.0) 2564 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:41.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:41 smithi151 bash[22565]: cluster 2024-09-10T18:44:40.608581+0000 mon.a (mon.0) 2565 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-09-10T18:44:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:44:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:41] "GET /metrics HTTP/1.1" 200 228660 "" "Prometheus/2.43.0" 2024-09-10T18:44:42.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:44:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:44:43.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:43 smithi151 bash[22565]: cluster 2024-09-10T18:44:41.980418+0000 mgr.y (mgr.24512) 411 : cluster [DBG] pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:44:43.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:43 smithi100 bash[31607]: cluster 2024-09-10T18:44:41.980418+0000 mgr.y (mgr.24512) 411 : cluster [DBG] pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:44:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:43 smithi100 bash[18597]: cluster 2024-09-10T18:44:41.980418+0000 mgr.y (mgr.24512) 411 : cluster [DBG] pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:44:44.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:44 smithi151 bash[22565]: audit 2024-09-10T18:44:42.444355+0000 mgr.y (mgr.24512) 412 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:44.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:44 smithi100 bash[31607]: audit 2024-09-10T18:44:42.444355+0000 mgr.y (mgr.24512) 412 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:44.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:44 smithi100 bash[18597]: audit 2024-09-10T18:44:42.444355+0000 mgr.y (mgr.24512) 412 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:44.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:44:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:45.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:45 smithi151 bash[22565]: cluster 2024-09-10T18:44:43.981219+0000 mgr.y (mgr.24512) 413 : cluster [DBG] pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:44:45.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:45 smithi151 bash[22565]: audit 2024-09-10T18:44:44.818488+0000 mon.b (mon.1) 283 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:45.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:45 smithi151 bash[22565]: audit 2024-09-10T18:44:44.820787+0000 mon.b (mon.1) 284 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-65"}]: dispatch 2024-09-10T18:44:45.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:45 smithi151 bash[22565]: audit 2024-09-10T18:44:44.832866+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:45.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:45 smithi151 bash[22565]: audit 2024-09-10T18:44:44.835089+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-65"}]: dispatch 2024-09-10T18:44:45.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[18597]: cluster 2024-09-10T18:44:43.981219+0000 mgr.y (mgr.24512) 413 : cluster [DBG] pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:44:45.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[18597]: audit 2024-09-10T18:44:44.818488+0000 mon.b (mon.1) 283 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:45.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[18597]: audit 2024-09-10T18:44:44.820787+0000 mon.b (mon.1) 284 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-65"}]: dispatch 2024-09-10T18:44:45.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[18597]: audit 2024-09-10T18:44:44.832866+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[18597]: audit 2024-09-10T18:44:44.835089+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-65"}]: dispatch 2024-09-10T18:44:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[31607]: cluster 2024-09-10T18:44:43.981219+0000 mgr.y (mgr.24512) 413 : cluster [DBG] pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:44:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[31607]: audit 2024-09-10T18:44:44.818488+0000 mon.b (mon.1) 283 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[31607]: audit 2024-09-10T18:44:44.820787+0000 mon.b (mon.1) 284 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-65"}]: dispatch 2024-09-10T18:44:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[31607]: audit 2024-09-10T18:44:44.832866+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:45 smithi100 bash[31607]: audit 2024-09-10T18:44:44.835089+0000 mon.a (mon.0) 2567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-65"}]: dispatch 2024-09-10T18:44:46.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:46 smithi151 bash[22565]: cluster 2024-09-10T18:44:45.078921+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-09-10T18:44:46.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:46 smithi151 bash[22565]: cluster 2024-09-10T18:44:45.983166+0000 mgr.y (mgr.24512) 414 : cluster [DBG] pgmap v630: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:46.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:46 smithi100 bash[18597]: cluster 2024-09-10T18:44:45.078921+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-09-10T18:44:46.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:46 smithi100 bash[18597]: cluster 2024-09-10T18:44:45.983166+0000 mgr.y (mgr.24512) 414 : cluster [DBG] pgmap v630: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:46.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:46 smithi100 bash[31607]: cluster 2024-09-10T18:44:45.078921+0000 mon.a (mon.0) 2568 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-09-10T18:44:46.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:46 smithi100 bash[31607]: cluster 2024-09-10T18:44:45.983166+0000 mgr.y (mgr.24512) 414 : cluster [DBG] pgmap v630: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:47.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:47 smithi151 bash[22565]: audit 2024-09-10T18:44:46.091634+0000 mon.b (mon.1) 285 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:47.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:47 smithi151 bash[22565]: cluster 2024-09-10T18:44:46.097115+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-09-10T18:44:47.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:47 smithi151 bash[22565]: audit 2024-09-10T18:44:46.111711+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:47.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:47 smithi151 bash[22565]: audit 2024-09-10T18:44:47.099197+0000 mon.a (mon.0) 2571 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:47.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:47 smithi151 bash[22565]: cluster 2024-09-10T18:44:47.099418+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-09-10T18:44:47.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[18597]: audit 2024-09-10T18:44:46.091634+0000 mon.b (mon.1) 285 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:47.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[18597]: cluster 2024-09-10T18:44:46.097115+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-09-10T18:44:47.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[18597]: audit 2024-09-10T18:44:46.111711+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:47.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[18597]: audit 2024-09-10T18:44:47.099197+0000 mon.a (mon.0) 2571 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:47.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[18597]: cluster 2024-09-10T18:44:47.099418+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-09-10T18:44:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[31607]: audit 2024-09-10T18:44:46.091634+0000 mon.b (mon.1) 285 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[31607]: cluster 2024-09-10T18:44:46.097115+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-09-10T18:44:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[31607]: audit 2024-09-10T18:44:46.111711+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[31607]: audit 2024-09-10T18:44:47.099197+0000 mon.a (mon.0) 2571 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:47 smithi100 bash[31607]: cluster 2024-09-10T18:44:47.099418+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-09-10T18:44:48.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:48 smithi151 bash[22565]: audit 2024-09-10T18:44:47.090422+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:48.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:48 smithi151 bash[22565]: cluster 2024-09-10T18:44:47.983950+0000 mgr.y (mgr.24512) 415 : cluster [DBG] pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:48 smithi100 bash[18597]: audit 2024-09-10T18:44:47.090422+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:48 smithi100 bash[18597]: cluster 2024-09-10T18:44:47.983950+0000 mgr.y (mgr.24512) 415 : cluster [DBG] pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:48.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:48 smithi100 bash[31607]: audit 2024-09-10T18:44:47.090422+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:48 smithi100 bash[31607]: cluster 2024-09-10T18:44:47.983950+0000 mgr.y (mgr.24512) 415 : cluster [DBG] pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:49.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:49 smithi151 bash[22565]: cluster 2024-09-10T18:44:48.133344+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-09-10T18:44:49.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:49 smithi151 bash[22565]: audit 2024-09-10T18:44:48.162091+0000 mon.b (mon.1) 287 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:49.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:49 smithi151 bash[22565]: audit 2024-09-10T18:44:48.164029+0000 mon.b (mon.1) 288 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-67"}]: dispatch 2024-09-10T18:44:49.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:49 smithi151 bash[22565]: audit 2024-09-10T18:44:48.176177+0000 mon.a (mon.0) 2574 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:49.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:49 smithi151 bash[22565]: audit 2024-09-10T18:44:48.177943+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-67"}]: dispatch 2024-09-10T18:44:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[18597]: cluster 2024-09-10T18:44:48.133344+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-09-10T18:44:50.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[18597]: audit 2024-09-10T18:44:48.162091+0000 mon.b (mon.1) 287 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[18597]: audit 2024-09-10T18:44:48.164029+0000 mon.b (mon.1) 288 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-67"}]: dispatch 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[18597]: audit 2024-09-10T18:44:48.176177+0000 mon.a (mon.0) 2574 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[18597]: audit 2024-09-10T18:44:48.177943+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-67"}]: dispatch 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[31607]: cluster 2024-09-10T18:44:48.133344+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[31607]: audit 2024-09-10T18:44:48.162091+0000 mon.b (mon.1) 287 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[31607]: audit 2024-09-10T18:44:48.164029+0000 mon.b (mon.1) 288 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-67"}]: dispatch 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[31607]: audit 2024-09-10T18:44:48.176177+0000 mon.a (mon.0) 2574 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:50.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:49 smithi100 bash[31607]: audit 2024-09-10T18:44:48.177943+0000 mon.a (mon.0) 2575 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-67"}]: dispatch 2024-09-10T18:44:50.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:50 smithi151 bash[22565]: cluster 2024-09-10T18:44:49.139970+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-09-10T18:44:50.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:50 smithi151 bash[22565]: cluster 2024-09-10T18:44:49.984699+0000 mgr.y (mgr.24512) 416 : cluster [DBG] pgmap v636: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:50.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:50 smithi151 bash[22565]: audit 2024-09-10T18:44:50.127359+0000 mon.b (mon.1) 289 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:50.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:50 smithi151 bash[22565]: cluster 2024-09-10T18:44:50.137652+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-09-10T18:44:50.926 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:50 smithi151 bash[22565]: audit 2024-09-10T18:44:50.142601+0000 mon.a (mon.0) 2578 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:51.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[18597]: cluster 2024-09-10T18:44:49.139970+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-09-10T18:44:51.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[18597]: cluster 2024-09-10T18:44:49.984699+0000 mgr.y (mgr.24512) 416 : cluster [DBG] pgmap v636: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:51.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[18597]: audit 2024-09-10T18:44:50.127359+0000 mon.b (mon.1) 289 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[18597]: cluster 2024-09-10T18:44:50.137652+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-09-10T18:44:51.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[18597]: audit 2024-09-10T18:44:50.142601+0000 mon.a (mon.0) 2578 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[31607]: cluster 2024-09-10T18:44:49.139970+0000 mon.a (mon.0) 2576 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-09-10T18:44:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[31607]: cluster 2024-09-10T18:44:49.984699+0000 mgr.y (mgr.24512) 416 : cluster [DBG] pgmap v636: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[31607]: audit 2024-09-10T18:44:50.127359+0000 mon.b (mon.1) 289 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[31607]: cluster 2024-09-10T18:44:50.137652+0000 mon.a (mon.0) 2577 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-09-10T18:44:51.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:50 smithi100 bash[31607]: audit 2024-09-10T18:44:50.142601+0000 mon.a (mon.0) 2578 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:44:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:51] "GET /metrics HTTP/1.1" 200 228660 "" "Prometheus/2.43.0" 2024-09-10T18:44:52.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: audit 2024-09-10T18:44:51.135154+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:52.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: audit 2024-09-10T18:44:51.144272+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:52.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: cluster 2024-09-10T18:44:51.145005+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-09-10T18:44:52.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: audit 2024-09-10T18:44:51.176045+0000 mon.b (mon.1) 291 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:52.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: audit 2024-09-10T18:44:51.177508+0000 mon.b (mon.1) 292 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-69"}]: dispatch 2024-09-10T18:44:52.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: audit 2024-09-10T18:44:51.189881+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:52.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: audit 2024-09-10T18:44:51.191303+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-69"}]: dispatch 2024-09-10T18:44:52.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[22565]: cluster 2024-09-10T18:44:51.985500+0000 mgr.y (mgr.24512) 417 : cluster [DBG] pgmap v639: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: audit 2024-09-10T18:44:51.135154+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: audit 2024-09-10T18:44:51.144272+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: cluster 2024-09-10T18:44:51.145005+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-09-10T18:44:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: audit 2024-09-10T18:44:51.176045+0000 mon.b (mon.1) 291 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: audit 2024-09-10T18:44:51.177508+0000 mon.b (mon.1) 292 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-69"}]: dispatch 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: audit 2024-09-10T18:44:51.189881+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: audit 2024-09-10T18:44:51.191303+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-69"}]: dispatch 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[18597]: cluster 2024-09-10T18:44:51.985500+0000 mgr.y (mgr.24512) 417 : cluster [DBG] pgmap v639: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: audit 2024-09-10T18:44:51.135154+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: audit 2024-09-10T18:44:51.144272+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: cluster 2024-09-10T18:44:51.145005+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: audit 2024-09-10T18:44:51.176045+0000 mon.b (mon.1) 291 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: audit 2024-09-10T18:44:51.177508+0000 mon.b (mon.1) 292 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-69"}]: dispatch 2024-09-10T18:44:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: audit 2024-09-10T18:44:51.189881+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:44:52.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: audit 2024-09-10T18:44:51.191303+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-69"}]: dispatch 2024-09-10T18:44:52.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:52 smithi100 bash[31607]: cluster 2024-09-10T18:44:51.985500+0000 mgr.y (mgr.24512) 417 : cluster [DBG] pgmap v639: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:52.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:44:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:44:53.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:53 smithi151 bash[22565]: cluster 2024-09-10T18:44:52.153882+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-09-10T18:44:53.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:53 smithi151 bash[22565]: audit 2024-09-10T18:44:52.452031+0000 mgr.y (mgr.24512) 418 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:53.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:53 smithi100 bash[31607]: cluster 2024-09-10T18:44:52.153882+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-09-10T18:44:53.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:53 smithi100 bash[31607]: audit 2024-09-10T18:44:52.452031+0000 mgr.y (mgr.24512) 418 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:53 smithi100 bash[18597]: cluster 2024-09-10T18:44:52.153882+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-09-10T18:44:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:53 smithi100 bash[18597]: audit 2024-09-10T18:44:52.452031+0000 mgr.y (mgr.24512) 418 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[18597]: audit 2024-09-10T18:44:53.166158+0000 mon.b (mon.1) 293 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[18597]: cluster 2024-09-10T18:44:53.170074+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-09-10T18:44:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[18597]: audit 2024-09-10T18:44:53.182726+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[18597]: cluster 2024-09-10T18:44:53.986522+0000 mgr.y (mgr.24512) 419 : cluster [DBG] pgmap v642: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[31607]: audit 2024-09-10T18:44:53.166158+0000 mon.b (mon.1) 293 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[31607]: cluster 2024-09-10T18:44:53.170074+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-09-10T18:44:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[31607]: audit 2024-09-10T18:44:53.182726+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:54 smithi100 bash[31607]: cluster 2024-09-10T18:44:53.986522+0000 mgr.y (mgr.24512) 419 : cluster [DBG] pgmap v642: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:54.551 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:54 smithi151 bash[22565]: audit 2024-09-10T18:44:53.166158+0000 mon.b (mon.1) 293 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:54.551 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:54 smithi151 bash[22565]: cluster 2024-09-10T18:44:53.170074+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-09-10T18:44:54.551 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:54 smithi151 bash[22565]: audit 2024-09-10T18:44:53.182726+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:54.551 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:54 smithi151 bash[22565]: cluster 2024-09-10T18:44:53.986522+0000 mgr.y (mgr.24512) 419 : cluster [DBG] pgmap v642: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:54.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:44:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:44:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:55.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: cluster 2024-09-10T18:44:54.165546+0000 mon.a (mon.0) 2586 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:55.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: audit 2024-09-10T18:44:54.167629+0000 mon.b (mon.1) 294 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:55.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: audit 2024-09-10T18:44:54.171854+0000 mon.b (mon.1) 295 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: audit 2024-09-10T18:44:54.178851+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: cluster 2024-09-10T18:44:54.179033+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: audit 2024-09-10T18:44:54.201971+0000 mon.a (mon.0) 2589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: audit 2024-09-10T18:44:54.331957+0000 mon.c (mon.2) 445 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: audit 2024-09-10T18:44:55.176623+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[18597]: cluster 2024-09-10T18:44:55.176821+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: cluster 2024-09-10T18:44:54.165546+0000 mon.a (mon.0) 2586 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: audit 2024-09-10T18:44:54.167629+0000 mon.b (mon.1) 294 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: audit 2024-09-10T18:44:54.171854+0000 mon.b (mon.1) 295 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: audit 2024-09-10T18:44:54.178851+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: cluster 2024-09-10T18:44:54.179033+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-09-10T18:44:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: audit 2024-09-10T18:44:54.201971+0000 mon.a (mon.0) 2589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: audit 2024-09-10T18:44:54.331957+0000 mon.c (mon.2) 445 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: audit 2024-09-10T18:44:55.176623+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:44:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:55 smithi100 bash[31607]: cluster 2024-09-10T18:44:55.176821+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-09-10T18:44:55.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: cluster 2024-09-10T18:44:54.165546+0000 mon.a (mon.0) 2586 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:55.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: audit 2024-09-10T18:44:54.167629+0000 mon.b (mon.1) 294 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:44:55.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: audit 2024-09-10T18:44:54.171854+0000 mon.b (mon.1) 295 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:55.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: audit 2024-09-10T18:44:54.178851+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:55.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: cluster 2024-09-10T18:44:54.179033+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-09-10T18:44:55.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: audit 2024-09-10T18:44:54.201971+0000 mon.a (mon.0) 2589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:44:55.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: audit 2024-09-10T18:44:54.331957+0000 mon.c (mon.2) 445 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:55.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: audit 2024-09-10T18:44:55.176623+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:44:55.676 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:55 smithi151 bash[22565]: cluster 2024-09-10T18:44:55.176821+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-09-10T18:44:56.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:56 smithi100 bash[18597]: cluster 2024-09-10T18:44:55.988735+0000 mgr.y (mgr.24512) 420 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T18:44:56.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:56 smithi100 bash[31607]: cluster 2024-09-10T18:44:55.988735+0000 mgr.y (mgr.24512) 420 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T18:44:56.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:56 smithi151 bash[22565]: cluster 2024-09-10T18:44:55.988735+0000 mgr.y (mgr.24512) 420 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T18:44:57.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:57 smithi100 bash[18597]: cluster 2024-09-10T18:44:56.212383+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-09-10T18:44:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:57 smithi100 bash[31607]: cluster 2024-09-10T18:44:56.212383+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-09-10T18:44:57.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:57 smithi151 bash[22565]: cluster 2024-09-10T18:44:56.212383+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-09-10T18:44:58.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:44:58 smithi100 bash[18597]: cluster 2024-09-10T18:44:57.989543+0000 mgr.y (mgr.24512) 421 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-10T18:44:58.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:44:58 smithi100 bash[31607]: cluster 2024-09-10T18:44:57.989543+0000 mgr.y (mgr.24512) 421 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-10T18:44:58.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:44:58 smithi151 bash[22565]: cluster 2024-09-10T18:44:57.989543+0000 mgr.y (mgr.24512) 421 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-10T18:45:01.344 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:01 smithi100 bash[18597]: cluster 2024-09-10T18:44:59.990327+0000 mgr.y (mgr.24512) 422 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:45:01.344 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:01 smithi100 bash[18597]: cluster 2024-09-10T18:45:00.608170+0000 mon.a (mon.0) 2593 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:01.345 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:01 smithi100 bash[31607]: cluster 2024-09-10T18:44:59.990327+0000 mgr.y (mgr.24512) 422 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:45:01.345 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:01 smithi100 bash[31607]: cluster 2024-09-10T18:45:00.608170+0000 mon.a (mon.0) 2593 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:01.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:01 smithi151 bash[22565]: cluster 2024-09-10T18:44:59.990327+0000 mgr.y (mgr.24512) 422 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-09-10T18:45:01.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:01 smithi151 bash[22565]: cluster 2024-09-10T18:45:00.608170+0000 mon.a (mon.0) 2593 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:01.735 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:45:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:01] "GET /metrics HTTP/1.1" 200 228624 "" "Prometheus/2.43.0" 2024-09-10T18:45:02.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:45:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:45:03.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:03 smithi151 bash[22565]: cluster 2024-09-10T18:45:01.992517+0000 mgr.y (mgr.24512) 423 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T18:45:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:03 smithi100 bash[18597]: cluster 2024-09-10T18:45:01.992517+0000 mgr.y (mgr.24512) 423 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T18:45:03.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:03 smithi100 bash[31607]: cluster 2024-09-10T18:45:01.992517+0000 mgr.y (mgr.24512) 423 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T18:45:04.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:04 smithi151 bash[22565]: audit 2024-09-10T18:45:02.463065+0000 mgr.y (mgr.24512) 424 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:04 smithi100 bash[18597]: audit 2024-09-10T18:45:02.463065+0000 mgr.y (mgr.24512) 424 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:04.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:04 smithi100 bash[31607]: audit 2024-09-10T18:45:02.463065+0000 mgr.y (mgr.24512) 424 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:04.925 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:45:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:05.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:05 smithi151 bash[22565]: cluster 2024-09-10T18:45:03.993303+0000 mgr.y (mgr.24512) 425 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-10T18:45:05.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:05 smithi100 bash[18597]: cluster 2024-09-10T18:45:03.993303+0000 mgr.y (mgr.24512) 425 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-10T18:45:05.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:05 smithi100 bash[31607]: cluster 2024-09-10T18:45:03.993303+0000 mgr.y (mgr.24512) 425 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-09-10T18:45:06.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:06 smithi151 bash[22565]: cluster 2024-09-10T18:45:05.995385+0000 mgr.y (mgr.24512) 426 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-09-10T18:45:06.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:06 smithi100 bash[18597]: cluster 2024-09-10T18:45:05.995385+0000 mgr.y (mgr.24512) 426 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-09-10T18:45:06.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:06 smithi100 bash[31607]: cluster 2024-09-10T18:45:05.995385+0000 mgr.y (mgr.24512) 426 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-09-10T18:45:09.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:09 smithi151 bash[22565]: cluster 2024-09-10T18:45:07.996207+0000 mgr.y (mgr.24512) 427 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 955 B/s rd, 347 B/s wr, 1 op/s 2024-09-10T18:45:09.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:09 smithi100 bash[18597]: cluster 2024-09-10T18:45:07.996207+0000 mgr.y (mgr.24512) 427 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 955 B/s rd, 347 B/s wr, 1 op/s 2024-09-10T18:45:09.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:09 smithi100 bash[31607]: cluster 2024-09-10T18:45:07.996207+0000 mgr.y (mgr.24512) 427 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 955 B/s rd, 347 B/s wr, 1 op/s 2024-09-10T18:45:10.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:10 smithi151 bash[22565]: audit 2024-09-10T18:45:09.332531+0000 mon.c (mon.2) 446 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:10 smithi100 bash[18597]: audit 2024-09-10T18:45:09.332531+0000 mon.c (mon.2) 446 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:10.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:10 smithi100 bash[31607]: audit 2024-09-10T18:45:09.332531+0000 mon.c (mon.2) 446 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:11.344 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:11 smithi100 bash[18597]: cluster 2024-09-10T18:45:09.997034+0000 mgr.y (mgr.24512) 428 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T18:45:11.345 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:11 smithi100 bash[31607]: cluster 2024-09-10T18:45:09.997034+0000 mgr.y (mgr.24512) 428 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T18:45:11.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:11 smithi151 bash[22565]: cluster 2024-09-10T18:45:09.997034+0000 mgr.y (mgr.24512) 428 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T18:45:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:45:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:11] "GET /metrics HTTP/1.1" 200 228587 "" "Prometheus/2.43.0" 2024-09-10T18:45:12.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:45:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:45:13.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:13 smithi151 bash[22565]: cluster 2024-09-10T18:45:11.999274+0000 mgr.y (mgr.24512) 429 : cluster [DBG] pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 596 B/s wr, 3 op/s 2024-09-10T18:45:13.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:13 smithi100 bash[18597]: cluster 2024-09-10T18:45:11.999274+0000 mgr.y (mgr.24512) 429 : cluster [DBG] pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 596 B/s wr, 3 op/s 2024-09-10T18:45:13.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:13 smithi100 bash[31607]: cluster 2024-09-10T18:45:11.999274+0000 mgr.y (mgr.24512) 429 : cluster [DBG] pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 596 B/s wr, 3 op/s 2024-09-10T18:45:14.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:14 smithi151 bash[22565]: audit 2024-09-10T18:45:12.471325+0000 mgr.y (mgr.24512) 430 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:14.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:14 smithi151 bash[22565]: cluster 2024-09-10T18:45:14.000093+0000 mgr.y (mgr.24512) 431 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:45:14.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:14 smithi100 bash[31607]: audit 2024-09-10T18:45:12.471325+0000 mgr.y (mgr.24512) 430 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:14.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:14 smithi100 bash[31607]: cluster 2024-09-10T18:45:14.000093+0000 mgr.y (mgr.24512) 431 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:45:14.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:14 smithi100 bash[18597]: audit 2024-09-10T18:45:12.471325+0000 mgr.y (mgr.24512) 430 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:14.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:14 smithi100 bash[18597]: cluster 2024-09-10T18:45:14.000093+0000 mgr.y (mgr.24512) 431 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:45:14.924 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:45:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:17.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:17 smithi151 bash[22565]: cluster 2024-09-10T18:45:16.002261+0000 mgr.y (mgr.24512) 432 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:45:17.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:17 smithi151 bash[22565]: audit 2024-09-10T18:45:16.279746+0000 mon.b (mon.1) 296 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:17.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:17 smithi151 bash[22565]: audit 2024-09-10T18:45:16.281989+0000 mon.b (mon.1) 297 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-71"}]: dispatch 2024-09-10T18:45:17.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:17 smithi151 bash[22565]: audit 2024-09-10T18:45:16.292847+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:17.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:17 smithi151 bash[22565]: audit 2024-09-10T18:45:16.295182+0000 mon.a (mon.0) 2595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-71"}]: dispatch 2024-09-10T18:45:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[18597]: cluster 2024-09-10T18:45:16.002261+0000 mgr.y (mgr.24512) 432 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[18597]: audit 2024-09-10T18:45:16.279746+0000 mon.b (mon.1) 296 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[18597]: audit 2024-09-10T18:45:16.281989+0000 mon.b (mon.1) 297 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-71"}]: dispatch 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[18597]: audit 2024-09-10T18:45:16.292847+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[18597]: audit 2024-09-10T18:45:16.295182+0000 mon.a (mon.0) 2595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-71"}]: dispatch 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[31607]: cluster 2024-09-10T18:45:16.002261+0000 mgr.y (mgr.24512) 432 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[31607]: audit 2024-09-10T18:45:16.279746+0000 mon.b (mon.1) 296 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[31607]: audit 2024-09-10T18:45:16.281989+0000 mon.b (mon.1) 297 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-71"}]: dispatch 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[31607]: audit 2024-09-10T18:45:16.292847+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:17 smithi100 bash[31607]: audit 2024-09-10T18:45:16.295182+0000 mon.a (mon.0) 2595 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-71"}]: dispatch 2024-09-10T18:45:18.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:18 smithi151 bash[22565]: cluster 2024-09-10T18:45:17.073637+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-09-10T18:45:18.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:18 smithi100 bash[18597]: cluster 2024-09-10T18:45:17.073637+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-09-10T18:45:18.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:18 smithi100 bash[31607]: cluster 2024-09-10T18:45:17.073637+0000 mon.a (mon.0) 2596 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-09-10T18:45:19.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:19 smithi151 bash[22565]: cluster 2024-09-10T18:45:18.002968+0000 mgr.y (mgr.24512) 433 : cluster [DBG] pgmap v658: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-09-10T18:45:19.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:19 smithi151 bash[22565]: audit 2024-09-10T18:45:18.079902+0000 mon.b (mon.1) 298 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:19.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:19 smithi151 bash[22565]: cluster 2024-09-10T18:45:18.090032+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-09-10T18:45:19.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:19 smithi151 bash[22565]: audit 2024-09-10T18:45:18.093302+0000 mon.a (mon.0) 2598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:19.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[18597]: cluster 2024-09-10T18:45:18.002968+0000 mgr.y (mgr.24512) 433 : cluster [DBG] pgmap v658: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-09-10T18:45:19.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[18597]: audit 2024-09-10T18:45:18.079902+0000 mon.b (mon.1) 298 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:19.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[18597]: cluster 2024-09-10T18:45:18.090032+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-09-10T18:45:19.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[18597]: audit 2024-09-10T18:45:18.093302+0000 mon.a (mon.0) 2598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:19.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[31607]: cluster 2024-09-10T18:45:18.002968+0000 mgr.y (mgr.24512) 433 : cluster [DBG] pgmap v658: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-09-10T18:45:19.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[31607]: audit 2024-09-10T18:45:18.079902+0000 mon.b (mon.1) 298 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:19.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[31607]: cluster 2024-09-10T18:45:18.090032+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-09-10T18:45:19.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:19 smithi100 bash[31607]: audit 2024-09-10T18:45:18.093302+0000 mon.a (mon.0) 2598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:20.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:20 smithi151 bash[22565]: audit 2024-09-10T18:45:19.086626+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:20.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:20 smithi151 bash[22565]: audit 2024-09-10T18:45:19.096089+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:20.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:20 smithi151 bash[22565]: cluster 2024-09-10T18:45:19.096460+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-09-10T18:45:20.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:20 smithi151 bash[22565]: cluster 2024-09-10T18:45:20.003746+0000 mgr.y (mgr.24512) 434 : cluster [DBG] pgmap v661: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:20.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[18597]: audit 2024-09-10T18:45:19.086626+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:20.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[18597]: audit 2024-09-10T18:45:19.096089+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[18597]: cluster 2024-09-10T18:45:19.096460+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-09-10T18:45:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[18597]: cluster 2024-09-10T18:45:20.003746+0000 mgr.y (mgr.24512) 434 : cluster [DBG] pgmap v661: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[31607]: audit 2024-09-10T18:45:19.086626+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[31607]: audit 2024-09-10T18:45:19.096089+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[31607]: cluster 2024-09-10T18:45:19.096460+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-09-10T18:45:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:20 smithi100 bash[31607]: cluster 2024-09-10T18:45:20.003746+0000 mgr.y (mgr.24512) 434 : cluster [DBG] pgmap v661: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:21.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:21 smithi151 bash[22565]: cluster 2024-09-10T18:45:20.120443+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-09-10T18:45:21.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:21 smithi100 bash[18597]: cluster 2024-09-10T18:45:20.120443+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-09-10T18:45:21.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:45:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:21] "GET /metrics HTTP/1.1" 200 228587 "" "Prometheus/2.43.0" 2024-09-10T18:45:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:21 smithi100 bash[31607]: cluster 2024-09-10T18:45:20.120443+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-09-10T18:45:22.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:22 smithi151 bash[22565]: cluster 2024-09-10T18:45:21.124615+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-09-10T18:45:22.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:22 smithi151 bash[22565]: cluster 2024-09-10T18:45:22.004554+0000 mgr.y (mgr.24512) 435 : cluster [DBG] pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:45:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:22 smithi100 bash[18597]: cluster 2024-09-10T18:45:21.124615+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-09-10T18:45:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:22 smithi100 bash[18597]: cluster 2024-09-10T18:45:22.004554+0000 mgr.y (mgr.24512) 435 : cluster [DBG] pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:45:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:22 smithi100 bash[31607]: cluster 2024-09-10T18:45:21.124615+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-09-10T18:45:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:22 smithi100 bash[31607]: cluster 2024-09-10T18:45:22.004554+0000 mgr.y (mgr.24512) 435 : cluster [DBG] pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:45:22.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:45:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:45:23.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:23 smithi151 bash[22565]: audit 2024-09-10T18:45:22.479040+0000 mgr.y (mgr.24512) 436 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:23.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:23 smithi100 bash[18597]: audit 2024-09-10T18:45:22.479040+0000 mgr.y (mgr.24512) 436 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:23.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:23 smithi100 bash[31607]: audit 2024-09-10T18:45:22.479040+0000 mgr.y (mgr.24512) 436 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:24.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:24 smithi151 bash[22565]: cluster 2024-09-10T18:45:24.005416+0000 mgr.y (mgr.24512) 437 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 864 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-09-10T18:45:24.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:24 smithi100 bash[18597]: cluster 2024-09-10T18:45:24.005416+0000 mgr.y (mgr.24512) 437 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 864 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-09-10T18:45:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:24 smithi100 bash[31607]: cluster 2024-09-10T18:45:24.005416+0000 mgr.y (mgr.24512) 437 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 864 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-09-10T18:45:24.924 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:45:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:25.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:25 smithi151 bash[22565]: audit 2024-09-10T18:45:24.333116+0000 mon.c (mon.2) 447 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:25.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:25 smithi100 bash[18597]: audit 2024-09-10T18:45:24.333116+0000 mon.c (mon.2) 447 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:25.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:25 smithi100 bash[31607]: audit 2024-09-10T18:45:24.333116+0000 mon.c (mon.2) 447 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:26.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:26 smithi151 bash[22565]: cluster 2024-09-10T18:45:26.007576+0000 mgr.y (mgr.24512) 438 : cluster [DBG] pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-10T18:45:26.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:26 smithi100 bash[18597]: cluster 2024-09-10T18:45:26.007576+0000 mgr.y (mgr.24512) 438 : cluster [DBG] pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-10T18:45:26.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:26 smithi100 bash[31607]: cluster 2024-09-10T18:45:26.007576+0000 mgr.y (mgr.24512) 438 : cluster [DBG] pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-10T18:45:29.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:29 smithi151 bash[22565]: cluster 2024-09-10T18:45:28.008389+0000 mgr.y (mgr.24512) 439 : cluster [DBG] pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-10T18:45:29.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:29 smithi100 bash[18597]: cluster 2024-09-10T18:45:28.008389+0000 mgr.y (mgr.24512) 439 : cluster [DBG] pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-10T18:45:29.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:29 smithi100 bash[31607]: cluster 2024-09-10T18:45:28.008389+0000 mgr.y (mgr.24512) 439 : cluster [DBG] pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-10T18:45:31.343 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:31 smithi100 bash[18597]: cluster 2024-09-10T18:45:30.009183+0000 mgr.y (mgr.24512) 440 : cluster [DBG] pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 724 B/s wr, 3 op/s 2024-09-10T18:45:31.343 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:31 smithi100 bash[31607]: cluster 2024-09-10T18:45:30.009183+0000 mgr.y (mgr.24512) 440 : cluster [DBG] pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 724 B/s wr, 3 op/s 2024-09-10T18:45:31.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:31 smithi151 bash[22565]: cluster 2024-09-10T18:45:30.009183+0000 mgr.y (mgr.24512) 440 : cluster [DBG] pgmap v668: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 724 B/s wr, 3 op/s 2024-09-10T18:45:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:45:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:31] "GET /metrics HTTP/1.1" 200 228643 "" "Prometheus/2.43.0" 2024-09-10T18:45:32.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:32 smithi151 bash[22565]: audit 2024-09-10T18:45:31.189573+0000 mon.b (mon.1) 300 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:32.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:32 smithi151 bash[22565]: audit 2024-09-10T18:45:31.192024+0000 mon.b (mon.1) 301 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-73"}]: dispatch 2024-09-10T18:45:32.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:32 smithi151 bash[22565]: audit 2024-09-10T18:45:31.202307+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:32.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:32 smithi151 bash[22565]: audit 2024-09-10T18:45:31.204627+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-73"}]: dispatch 2024-09-10T18:45:32.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[31607]: audit 2024-09-10T18:45:31.189573+0000 mon.b (mon.1) 300 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[31607]: audit 2024-09-10T18:45:31.192024+0000 mon.b (mon.1) 301 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-73"}]: dispatch 2024-09-10T18:45:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[31607]: audit 2024-09-10T18:45:31.202307+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[31607]: audit 2024-09-10T18:45:31.204627+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-73"}]: dispatch 2024-09-10T18:45:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[18597]: audit 2024-09-10T18:45:31.189573+0000 mon.b (mon.1) 300 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[18597]: audit 2024-09-10T18:45:31.192024+0000 mon.b (mon.1) 301 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-73"}]: dispatch 2024-09-10T18:45:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[18597]: audit 2024-09-10T18:45:31.202307+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:32 smithi100 bash[18597]: audit 2024-09-10T18:45:31.204627+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-73"}]: dispatch 2024-09-10T18:45:32.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:45:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:45:33.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:33 smithi151 bash[22565]: cluster 2024-09-10T18:45:32.011394+0000 mgr.y (mgr.24512) 441 : cluster [DBG] pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 94 B/s wr, 2 op/s 2024-09-10T18:45:33.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:33 smithi151 bash[22565]: cluster 2024-09-10T18:45:32.084513+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-09-10T18:45:33.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:33 smithi151 bash[22565]: audit 2024-09-10T18:45:32.486725+0000 mgr.y (mgr.24512) 442 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:33.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:33 smithi100 bash[18597]: cluster 2024-09-10T18:45:32.011394+0000 mgr.y (mgr.24512) 441 : cluster [DBG] pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 94 B/s wr, 2 op/s 2024-09-10T18:45:33.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:33 smithi100 bash[18597]: cluster 2024-09-10T18:45:32.084513+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-09-10T18:45:33.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:33 smithi100 bash[18597]: audit 2024-09-10T18:45:32.486725+0000 mgr.y (mgr.24512) 442 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:33.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:33 smithi100 bash[31607]: cluster 2024-09-10T18:45:32.011394+0000 mgr.y (mgr.24512) 441 : cluster [DBG] pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 94 B/s wr, 2 op/s 2024-09-10T18:45:33.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:33 smithi100 bash[31607]: cluster 2024-09-10T18:45:32.084513+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-09-10T18:45:33.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:33 smithi100 bash[31607]: audit 2024-09-10T18:45:32.486725+0000 mgr.y (mgr.24512) 442 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:34 smithi151 bash[22565]: audit 2024-09-10T18:45:33.087929+0000 mon.b (mon.1) 302 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:34 smithi151 bash[22565]: cluster 2024-09-10T18:45:33.096576+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-09-10T18:45:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:34 smithi151 bash[22565]: audit 2024-09-10T18:45:33.102294+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:34 smithi151 bash[22565]: cluster 2024-09-10T18:45:34.012261+0000 mgr.y (mgr.24512) 443 : cluster [DBG] pgmap v672: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:34.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[18597]: audit 2024-09-10T18:45:33.087929+0000 mon.b (mon.1) 302 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:34.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[18597]: cluster 2024-09-10T18:45:33.096576+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-09-10T18:45:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[18597]: audit 2024-09-10T18:45:33.102294+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[18597]: cluster 2024-09-10T18:45:34.012261+0000 mgr.y (mgr.24512) 443 : cluster [DBG] pgmap v672: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[31607]: audit 2024-09-10T18:45:33.087929+0000 mon.b (mon.1) 302 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[31607]: cluster 2024-09-10T18:45:33.096576+0000 mon.a (mon.0) 2606 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-09-10T18:45:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[31607]: audit 2024-09-10T18:45:33.102294+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:34 smithi100 bash[31607]: cluster 2024-09-10T18:45:34.012261+0000 mgr.y (mgr.24512) 443 : cluster [DBG] pgmap v672: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:34.924 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:45:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:35.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:35 smithi151 bash[22565]: cluster 2024-09-10T18:45:34.091611+0000 mon.a (mon.0) 2608 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:35.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:35 smithi151 bash[22565]: audit 2024-09-10T18:45:34.106438+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:35.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:35 smithi151 bash[22565]: audit 2024-09-10T18:45:34.115379+0000 mon.a (mon.0) 2609 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:35.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:35 smithi151 bash[22565]: cluster 2024-09-10T18:45:34.115887+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-09-10T18:45:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[18597]: cluster 2024-09-10T18:45:34.091611+0000 mon.a (mon.0) 2608 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[18597]: audit 2024-09-10T18:45:34.106438+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:35.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[18597]: audit 2024-09-10T18:45:34.115379+0000 mon.a (mon.0) 2609 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:35.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[18597]: cluster 2024-09-10T18:45:34.115887+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-09-10T18:45:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[31607]: cluster 2024-09-10T18:45:34.091611+0000 mon.a (mon.0) 2608 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[31607]: audit 2024-09-10T18:45:34.106438+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[31607]: audit 2024-09-10T18:45:34.115379+0000 mon.a (mon.0) 2609 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:35 smithi100 bash[31607]: cluster 2024-09-10T18:45:34.115887+0000 mon.a (mon.0) 2610 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-09-10T18:45:36.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:36 smithi151 bash[22565]: cluster 2024-09-10T18:45:35.119049+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-09-10T18:45:36.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:36 smithi151 bash[22565]: cluster 2024-09-10T18:45:36.014491+0000 mgr.y (mgr.24512) 444 : cluster [DBG] pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:36.424 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:45:36 smithi151 bash[26435]: debug 2024-09-10T18:45:36.158+0000 7fc38dde7640 -1 snap_mapper.add_oid found existing snaps mapped on 65:8b7e713b:test-rados-api-smithi100-72764-76::foo:28, removing 2024-09-10T18:45:36.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:36 smithi100 bash[18597]: cluster 2024-09-10T18:45:35.119049+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-09-10T18:45:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:36 smithi100 bash[18597]: cluster 2024-09-10T18:45:36.014491+0000 mgr.y (mgr.24512) 444 : cluster [DBG] pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:36 smithi100 bash[31607]: cluster 2024-09-10T18:45:35.119049+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-09-10T18:45:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:36 smithi100 bash[31607]: cluster 2024-09-10T18:45:36.014491+0000 mgr.y (mgr.24512) 444 : cluster [DBG] pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:36.544 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 10 18:45:36 smithi100 bash[42533]: debug 2024-09-10T18:45:36.172+0000 7ff7b8395640 -1 snap_mapper.add_oid found existing snaps mapped on 65:8b7e713b:test-rados-api-smithi100-72764-76::foo:28, removing 2024-09-10T18:45:36.545 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:45:36 smithi100 bash[58092]: debug 2024-09-10T18:45:36.172+0000 7fabd05cd640 -1 snap_mapper.add_oid found existing snaps mapped on 65:8b7e713b:test-rados-api-smithi100-72764-76::foo:28, removing 2024-09-10T18:45:37.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:37 smithi151 bash[22565]: cluster 2024-09-10T18:45:36.129689+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-09-10T18:45:37.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:37 smithi151 bash[22565]: audit 2024-09-10T18:45:36.212823+0000 mon.b (mon.1) 304 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:37.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:37 smithi151 bash[22565]: audit 2024-09-10T18:45:36.215153+0000 mon.b (mon.1) 305 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-75"}]: dispatch 2024-09-10T18:45:37.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:37 smithi151 bash[22565]: audit 2024-09-10T18:45:36.225497+0000 mon.a (mon.0) 2613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:37.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:37 smithi151 bash[22565]: audit 2024-09-10T18:45:36.227660+0000 mon.a (mon.0) 2614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-75"}]: dispatch 2024-09-10T18:45:37.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[18597]: cluster 2024-09-10T18:45:36.129689+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[18597]: audit 2024-09-10T18:45:36.212823+0000 mon.b (mon.1) 304 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[18597]: audit 2024-09-10T18:45:36.215153+0000 mon.b (mon.1) 305 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-75"}]: dispatch 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[18597]: audit 2024-09-10T18:45:36.225497+0000 mon.a (mon.0) 2613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[18597]: audit 2024-09-10T18:45:36.227660+0000 mon.a (mon.0) 2614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-75"}]: dispatch 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[31607]: cluster 2024-09-10T18:45:36.129689+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[31607]: audit 2024-09-10T18:45:36.212823+0000 mon.b (mon.1) 304 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[31607]: audit 2024-09-10T18:45:36.215153+0000 mon.b (mon.1) 305 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-75"}]: dispatch 2024-09-10T18:45:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[31607]: audit 2024-09-10T18:45:36.225497+0000 mon.a (mon.0) 2613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:37.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:37 smithi100 bash[31607]: audit 2024-09-10T18:45:36.227660+0000 mon.a (mon.0) 2614 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-75"}]: dispatch 2024-09-10T18:45:38.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:38 smithi151 bash[22565]: cluster 2024-09-10T18:45:37.138644+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-09-10T18:45:38.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:38 smithi151 bash[22565]: cluster 2024-09-10T18:45:38.015316+0000 mgr.y (mgr.24512) 445 : cluster [DBG] pgmap v678: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:45:38.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:38 smithi100 bash[18597]: cluster 2024-09-10T18:45:37.138644+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-09-10T18:45:38.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:38 smithi100 bash[18597]: cluster 2024-09-10T18:45:38.015316+0000 mgr.y (mgr.24512) 445 : cluster [DBG] pgmap v678: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:45:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:38 smithi100 bash[31607]: cluster 2024-09-10T18:45:37.138644+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-09-10T18:45:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:38 smithi100 bash[31607]: cluster 2024-09-10T18:45:38.015316+0000 mgr.y (mgr.24512) 445 : cluster [DBG] pgmap v678: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:45:39.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:39 smithi151 bash[22565]: audit 2024-09-10T18:45:38.148916+0000 mon.b (mon.1) 306 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:39.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:39 smithi151 bash[22565]: cluster 2024-09-10T18:45:38.157031+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-09-10T18:45:39.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:39 smithi151 bash[22565]: audit 2024-09-10T18:45:38.161197+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:39.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:39 smithi151 bash[22565]: audit 2024-09-10T18:45:38.761539+0000 mon.c (mon.2) 448 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:39.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:39 smithi151 bash[22565]: audit 2024-09-10T18:45:39.069352+0000 mon.a (mon.0) 2618 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:39.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:39 smithi151 bash[22565]: audit 2024-09-10T18:45:39.084993+0000 mon.a (mon.0) 2619 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:39.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[18597]: audit 2024-09-10T18:45:38.148916+0000 mon.b (mon.1) 306 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:39.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[18597]: cluster 2024-09-10T18:45:38.157031+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-09-10T18:45:39.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[18597]: audit 2024-09-10T18:45:38.161197+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:39.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[18597]: audit 2024-09-10T18:45:38.761539+0000 mon.c (mon.2) 448 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[18597]: audit 2024-09-10T18:45:39.069352+0000 mon.a (mon.0) 2618 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[18597]: audit 2024-09-10T18:45:39.084993+0000 mon.a (mon.0) 2619 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[31607]: audit 2024-09-10T18:45:38.148916+0000 mon.b (mon.1) 306 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[31607]: cluster 2024-09-10T18:45:38.157031+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[31607]: audit 2024-09-10T18:45:38.161197+0000 mon.a (mon.0) 2617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[31607]: audit 2024-09-10T18:45:38.761539+0000 mon.c (mon.2) 448 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[31607]: audit 2024-09-10T18:45:39.069352+0000 mon.a (mon.0) 2618 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:39 smithi100 bash[31607]: audit 2024-09-10T18:45:39.084993+0000 mon.a (mon.0) 2619 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.424 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[33542]: debug 2024-09-10T18:45:40.194+0000 7f3439bd9640 -1 snap_mapper.add_oid found existing snaps mapped on 65:f28f426d:test-rados-api-smithi100-72764-78::foo:29, removing 2024-09-10T18:45:40.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.151599+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.155492+0000 mon.c (mon.2) 449 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.161232+0000 mon.a (mon.0) 2620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: cluster 2024-09-10T18:45:39.161437+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.163385+0000 mon.a (mon.0) 2622 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.175937+0000 mon.c (mon.2) 450 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.176645+0000 mon.a (mon.0) 2623 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.181892+0000 mon.c (mon.2) 451 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.182538+0000 mon.a (mon.0) 2624 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.184394+0000 mon.c (mon.2) 452 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.185022+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: cephadm 2024-09-10T18:45:39.186888+0000 mgr.y (mgr.24512) 446 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 3293M 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.193765+0000 mon.a (mon.0) 2626 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.333269+0000 mon.c (mon.2) 453 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.529588+0000 mon.c (mon.2) 454 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.530219+0000 mon.a (mon.0) 2627 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.532430+0000 mon.c (mon.2) 455 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.533058+0000 mon.a (mon.0) 2628 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.534992+0000 mon.c (mon.2) 456 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.535513+0000 mon.a (mon.0) 2629 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.537433+0000 mon.c (mon.2) 457 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.537997+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: cephadm 2024-09-10T18:45:39.539484+0000 mgr.y (mgr.24512) 447 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 3293M 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.547919+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.426 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.549898+0000 mon.c (mon.2) 458 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:40.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.551219+0000 mon.c (mon.2) 459 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:40.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: audit 2024-09-10T18:45:39.559507+0000 mon.a (mon.0) 2632 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.427 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:40 smithi151 bash[22565]: cluster 2024-09-10T18:45:40.016160+0000 mgr.y (mgr.24512) 448 : cluster [DBG] pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:40.543 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[58092]: debug 2024-09-10T18:45:40.208+0000 7fabd75db640 -1 snap_mapper.add_oid found existing snaps mapped on 65:f28f426d:test-rados-api-smithi100-72764-78::foo:29, removing 2024-09-10T18:45:40.544 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[34936]: debug 2024-09-10T18:45:40.208+0000 7fc4c0b47640 -1 snap_mapper.add_oid found existing snaps mapped on 65:f28f426d:test-rados-api-smithi100-72764-78::foo:29, removing 2024-09-10T18:45:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.151599+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.155492+0000 mon.c (mon.2) 449 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.161232+0000 mon.a (mon.0) 2620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: cluster 2024-09-10T18:45:39.161437+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.163385+0000 mon.a (mon.0) 2622 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.175937+0000 mon.c (mon.2) 450 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.176645+0000 mon.a (mon.0) 2623 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.181892+0000 mon.c (mon.2) 451 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.182538+0000 mon.a (mon.0) 2624 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.184394+0000 mon.c (mon.2) 452 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.185022+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: cephadm 2024-09-10T18:45:39.186888+0000 mgr.y (mgr.24512) 446 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 3293M 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.193765+0000 mon.a (mon.0) 2626 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.333269+0000 mon.c (mon.2) 453 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.529588+0000 mon.c (mon.2) 454 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.530219+0000 mon.a (mon.0) 2627 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.532430+0000 mon.c (mon.2) 455 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.151599+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.155492+0000 mon.c (mon.2) 449 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.161232+0000 mon.a (mon.0) 2620 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: cluster 2024-09-10T18:45:39.161437+0000 mon.a (mon.0) 2621 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.163385+0000 mon.a (mon.0) 2622 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.175937+0000 mon.c (mon.2) 450 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.176645+0000 mon.a (mon.0) 2623 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.181892+0000 mon.c (mon.2) 451 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.182538+0000 mon.a (mon.0) 2624 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.184394+0000 mon.c (mon.2) 452 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.185022+0000 mon.a (mon.0) 2625 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: cephadm 2024-09-10T18:45:39.186888+0000 mgr.y (mgr.24512) 446 : cephadm [INF] Adjusting osd_memory_target on smithi100 to 3293M 2024-09-10T18:45:40.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.193765+0000 mon.a (mon.0) 2626 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.333269+0000 mon.c (mon.2) 453 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.529588+0000 mon.c (mon.2) 454 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.530219+0000 mon.a (mon.0) 2627 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.532430+0000 mon.c (mon.2) 455 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.533058+0000 mon.a (mon.0) 2628 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.534992+0000 mon.c (mon.2) 456 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.535513+0000 mon.a (mon.0) 2629 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.537433+0000 mon.c (mon.2) 457 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.537997+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: cephadm 2024-09-10T18:45:39.539484+0000 mgr.y (mgr.24512) 447 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 3293M 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.547919+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.549898+0000 mon.c (mon.2) 458 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.551219+0000 mon.c (mon.2) 459 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: audit 2024-09-10T18:45:39.559507+0000 mon.a (mon.0) 2632 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[31607]: cluster 2024-09-10T18:45:40.016160+0000 mgr.y (mgr.24512) 448 : cluster [DBG] pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.533058+0000 mon.a (mon.0) 2628 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.549 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.534992+0000 mon.c (mon.2) 456 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.535513+0000 mon.a (mon.0) 2629 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.537433+0000 mon.c (mon.2) 457 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.537997+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24512 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: cephadm 2024-09-10T18:45:39.539484+0000 mgr.y (mgr.24512) 447 : cephadm [INF] Adjusting osd_memory_target on smithi151 to 3293M 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.547919+0000 mon.a (mon.0) 2631 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.549898+0000 mon.c (mon.2) 458 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.551219+0000 mon.c (mon.2) 459 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: audit 2024-09-10T18:45:39.559507+0000 mon.a (mon.0) 2632 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:45:40.550 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:40 smithi100 bash[18597]: cluster 2024-09-10T18:45:40.016160+0000 mgr.y (mgr.24512) 448 : cluster [DBG] pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:41.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:41] "GET /metrics HTTP/1.1" 200 228605 "" "Prometheus/2.43.0" 2024-09-10T18:45:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[18597]: cluster 2024-09-10T18:45:40.188287+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-09-10T18:45:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[18597]: audit 2024-09-10T18:45:40.223364+0000 mon.b (mon.1) 308 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[18597]: audit 2024-09-10T18:45:40.225405+0000 mon.b (mon.1) 309 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-77"}]: dispatch 2024-09-10T18:45:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[18597]: audit 2024-09-10T18:45:40.235813+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[18597]: audit 2024-09-10T18:45:40.237667+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-77"}]: dispatch 2024-09-10T18:45:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[18597]: cluster 2024-09-10T18:45:40.619385+0000 mon.a (mon.0) 2636 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[31607]: cluster 2024-09-10T18:45:40.188287+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-09-10T18:45:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[31607]: audit 2024-09-10T18:45:40.223364+0000 mon.b (mon.1) 308 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[31607]: audit 2024-09-10T18:45:40.225405+0000 mon.b (mon.1) 309 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-77"}]: dispatch 2024-09-10T18:45:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[31607]: audit 2024-09-10T18:45:40.235813+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:41.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[31607]: audit 2024-09-10T18:45:40.237667+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-77"}]: dispatch 2024-09-10T18:45:41.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:41 smithi100 bash[31607]: cluster 2024-09-10T18:45:40.619385+0000 mon.a (mon.0) 2636 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:41.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:41 smithi151 bash[22565]: cluster 2024-09-10T18:45:40.188287+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-09-10T18:45:41.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:41 smithi151 bash[22565]: audit 2024-09-10T18:45:40.223364+0000 mon.b (mon.1) 308 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:41.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:41 smithi151 bash[22565]: audit 2024-09-10T18:45:40.225405+0000 mon.b (mon.1) 309 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-77"}]: dispatch 2024-09-10T18:45:41.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:41 smithi151 bash[22565]: audit 2024-09-10T18:45:40.235813+0000 mon.a (mon.0) 2634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:45:41.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:41 smithi151 bash[22565]: audit 2024-09-10T18:45:40.237667+0000 mon.a (mon.0) 2635 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-77"}]: dispatch 2024-09-10T18:45:41.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:41 smithi151 bash[22565]: cluster 2024-09-10T18:45:40.619385+0000 mon.a (mon.0) 2636 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:42.498 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:42 smithi151 bash[22565]: cluster 2024-09-10T18:45:41.192580+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-09-10T18:45:42.498 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:42 smithi151 bash[22565]: cluster 2024-09-10T18:45:42.017118+0000 mgr.y (mgr.24512) 449 : cluster [DBG] pgmap v684: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T18:45:42.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:42 smithi100 bash[18597]: cluster 2024-09-10T18:45:41.192580+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-09-10T18:45:42.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:42 smithi100 bash[18597]: cluster 2024-09-10T18:45:42.017118+0000 mgr.y (mgr.24512) 449 : cluster [DBG] pgmap v684: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T18:45:42.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:42 smithi100 bash[31607]: cluster 2024-09-10T18:45:41.192580+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-09-10T18:45:42.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:42 smithi100 bash[31607]: cluster 2024-09-10T18:45:42.017118+0000 mgr.y (mgr.24512) 449 : cluster [DBG] pgmap v684: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T18:45:42.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:45:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:45:43.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[18597]: audit 2024-09-10T18:45:42.220857+0000 mon.b (mon.1) 310 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[18597]: cluster 2024-09-10T18:45:42.222357+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-09-10T18:45:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[18597]: audit 2024-09-10T18:45:42.242133+0000 mon.a (mon.0) 2639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[18597]: audit 2024-09-10T18:45:42.497767+0000 mgr.y (mgr.24512) 450 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[31607]: audit 2024-09-10T18:45:42.220857+0000 mon.b (mon.1) 310 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[31607]: cluster 2024-09-10T18:45:42.222357+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-09-10T18:45:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[31607]: audit 2024-09-10T18:45:42.242133+0000 mon.a (mon.0) 2639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:43 smithi100 bash[31607]: audit 2024-09-10T18:45:42.497767+0000 mgr.y (mgr.24512) 450 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:43.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:43 smithi151 bash[22565]: audit 2024-09-10T18:45:42.220857+0000 mon.b (mon.1) 310 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:43.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:43 smithi151 bash[22565]: cluster 2024-09-10T18:45:42.222357+0000 mon.a (mon.0) 2638 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-09-10T18:45:43.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:43 smithi151 bash[22565]: audit 2024-09-10T18:45:42.242133+0000 mon.a (mon.0) 2639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:43.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:43 smithi151 bash[22565]: audit 2024-09-10T18:45:42.497767+0000 mgr.y (mgr.24512) 450 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:44.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[31607]: audit 2024-09-10T18:45:43.221618+0000 mon.b (mon.1) 311 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[31607]: audit 2024-09-10T18:45:43.228248+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[31607]: cluster 2024-09-10T18:45:43.228466+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-09-10T18:45:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[31607]: cluster 2024-09-10T18:45:44.018106+0000 mgr.y (mgr.24512) 451 : cluster [DBG] pgmap v687: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T18:45:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[18597]: audit 2024-09-10T18:45:43.221618+0000 mon.b (mon.1) 311 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[18597]: audit 2024-09-10T18:45:43.228248+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[18597]: cluster 2024-09-10T18:45:43.228466+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-09-10T18:45:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:44 smithi100 bash[18597]: cluster 2024-09-10T18:45:44.018106+0000 mgr.y (mgr.24512) 451 : cluster [DBG] pgmap v687: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T18:45:44.548 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:44 smithi151 bash[22565]: audit 2024-09-10T18:45:43.221618+0000 mon.b (mon.1) 311 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:44.548 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:44 smithi151 bash[22565]: audit 2024-09-10T18:45:43.228248+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:44.548 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:44 smithi151 bash[22565]: cluster 2024-09-10T18:45:43.228466+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-09-10T18:45:44.549 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:44 smithi151 bash[22565]: cluster 2024-09-10T18:45:44.018106+0000 mgr.y (mgr.24512) 451 : cluster [DBG] pgmap v687: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T18:45:44.923 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:45:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:45.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:45 smithi100 bash[18597]: cluster 2024-09-10T18:45:44.240031+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-09-10T18:45:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:45 smithi100 bash[31607]: cluster 2024-09-10T18:45:44.240031+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-09-10T18:45:45.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:45 smithi151 bash[22565]: cluster 2024-09-10T18:45:44.240031+0000 mon.a (mon.0) 2642 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[18597]: cluster 2024-09-10T18:45:45.247432+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[18597]: audit 2024-09-10T18:45:45.258800+0000 mon.b (mon.1) 312 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1e"}]: dispatch 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[18597]: audit 2024-09-10T18:45:45.270603+0000 mgr.y (mgr.24512) 452 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1e"}]: dispatch 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[18597]: cluster 2024-09-10T18:45:46.020234+0000 mgr.y (mgr.24512) 453 : cluster [DBG] pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[31607]: cluster 2024-09-10T18:45:45.247432+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[31607]: audit 2024-09-10T18:45:45.258800+0000 mon.b (mon.1) 312 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1e"}]: dispatch 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[31607]: audit 2024-09-10T18:45:45.270603+0000 mgr.y (mgr.24512) 452 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1e"}]: dispatch 2024-09-10T18:45:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:46 smithi100 bash[31607]: cluster 2024-09-10T18:45:46.020234+0000 mgr.y (mgr.24512) 453 : cluster [DBG] pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:45:46.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:46 smithi151 bash[22565]: cluster 2024-09-10T18:45:45.247432+0000 mon.a (mon.0) 2643 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-09-10T18:45:46.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:46 smithi151 bash[22565]: audit 2024-09-10T18:45:45.258800+0000 mon.b (mon.1) 312 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1e"}]: dispatch 2024-09-10T18:45:46.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:46 smithi151 bash[22565]: audit 2024-09-10T18:45:45.270603+0000 mgr.y (mgr.24512) 452 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.1e"}]: dispatch 2024-09-10T18:45:46.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:46 smithi151 bash[22565]: cluster 2024-09-10T18:45:46.020234+0000 mgr.y (mgr.24512) 453 : cluster [DBG] pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:45:48.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:48 smithi151 bash[22565]: cluster 2024-09-10T18:45:48.021062+0000 mgr.y (mgr.24512) 454 : cluster [DBG] pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 882 B/s wr, 2 op/s 2024-09-10T18:45:48.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:48 smithi100 bash[18597]: cluster 2024-09-10T18:45:48.021062+0000 mgr.y (mgr.24512) 454 : cluster [DBG] pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 882 B/s wr, 2 op/s 2024-09-10T18:45:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:48 smithi100 bash[31607]: cluster 2024-09-10T18:45:48.021062+0000 mgr.y (mgr.24512) 454 : cluster [DBG] pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 882 B/s wr, 2 op/s 2024-09-10T18:45:50.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:50 smithi151 bash[22565]: cluster 2024-09-10T18:45:50.021904+0000 mgr.y (mgr.24512) 455 : cluster [DBG] pgmap v692: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 753 B/s rd, 753 B/s wr, 2 op/s 2024-09-10T18:45:50.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:50 smithi100 bash[18597]: cluster 2024-09-10T18:45:50.021904+0000 mgr.y (mgr.24512) 455 : cluster [DBG] pgmap v692: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 753 B/s rd, 753 B/s wr, 2 op/s 2024-09-10T18:45:50.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:50 smithi100 bash[31607]: cluster 2024-09-10T18:45:50.021904+0000 mgr.y (mgr.24512) 455 : cluster [DBG] pgmap v692: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 753 B/s rd, 753 B/s wr, 2 op/s 2024-09-10T18:45:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:45:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:51] "GET /metrics HTTP/1.1" 200 228605 "" "Prometheus/2.43.0" 2024-09-10T18:45:52.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:52 smithi151 bash[22565]: cluster 2024-09-10T18:45:52.024114+0000 mgr.y (mgr.24512) 456 : cluster [DBG] pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-10T18:45:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:52 smithi100 bash[18597]: cluster 2024-09-10T18:45:52.024114+0000 mgr.y (mgr.24512) 456 : cluster [DBG] pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-10T18:45:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:52 smithi100 bash[31607]: cluster 2024-09-10T18:45:52.024114+0000 mgr.y (mgr.24512) 456 : cluster [DBG] pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-10T18:45:52.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:45:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:45:53.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:53 smithi151 bash[22565]: audit 2024-09-10T18:45:52.508848+0000 mgr.y (mgr.24512) 457 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:53.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:53 smithi100 bash[18597]: audit 2024-09-10T18:45:52.508848+0000 mgr.y (mgr.24512) 457 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:53.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:53 smithi100 bash[31607]: audit 2024-09-10T18:45:52.508848+0000 mgr.y (mgr.24512) 457 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:54.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:54 smithi151 bash[22565]: cluster 2024-09-10T18:45:54.024973+0000 mgr.y (mgr.24512) 458 : cluster [DBG] pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 522 B/s wr, 2 op/s 2024-09-10T18:45:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:54 smithi100 bash[18597]: cluster 2024-09-10T18:45:54.024973+0000 mgr.y (mgr.24512) 458 : cluster [DBG] pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 522 B/s wr, 2 op/s 2024-09-10T18:45:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:54 smithi100 bash[31607]: cluster 2024-09-10T18:45:54.024973+0000 mgr.y (mgr.24512) 458 : cluster [DBG] pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 522 B/s wr, 2 op/s 2024-09-10T18:45:54.923 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:45:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:45:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:55.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:55 smithi151 bash[22565]: cluster 2024-09-10T18:45:54.075885+0000 osd.7 (osd.7) 13 : cluster [DBG] 223.1e deep-scrub starts 2024-09-10T18:45:55.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:55 smithi151 bash[22565]: cluster 2024-09-10T18:45:54.081167+0000 osd.7 (osd.7) 14 : cluster [DBG] 223.1e deep-scrub ok 2024-09-10T18:45:55.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:55 smithi151 bash[22565]: audit 2024-09-10T18:45:54.333767+0000 mon.c (mon.2) 460 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:55 smithi100 bash[31607]: cluster 2024-09-10T18:45:54.075885+0000 osd.7 (osd.7) 13 : cluster [DBG] 223.1e deep-scrub starts 2024-09-10T18:45:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:55 smithi100 bash[31607]: cluster 2024-09-10T18:45:54.081167+0000 osd.7 (osd.7) 14 : cluster [DBG] 223.1e deep-scrub ok 2024-09-10T18:45:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:55 smithi100 bash[31607]: audit 2024-09-10T18:45:54.333767+0000 mon.c (mon.2) 460 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:55 smithi100 bash[18597]: cluster 2024-09-10T18:45:54.075885+0000 osd.7 (osd.7) 13 : cluster [DBG] 223.1e deep-scrub starts 2024-09-10T18:45:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:55 smithi100 bash[18597]: cluster 2024-09-10T18:45:54.081167+0000 osd.7 (osd.7) 14 : cluster [DBG] 223.1e deep-scrub ok 2024-09-10T18:45:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:55 smithi100 bash[18597]: audit 2024-09-10T18:45:54.333767+0000 mon.c (mon.2) 460 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:56.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:56 smithi151 bash[22565]: cluster 2024-09-10T18:45:56.027115+0000 mgr.y (mgr.24512) 459 : cluster [DBG] pgmap v695: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 474 B/s wr, 2 op/s 2024-09-10T18:45:56.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:56 smithi100 bash[18597]: cluster 2024-09-10T18:45:56.027115+0000 mgr.y (mgr.24512) 459 : cluster [DBG] pgmap v695: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 474 B/s wr, 2 op/s 2024-09-10T18:45:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:56 smithi100 bash[31607]: cluster 2024-09-10T18:45:56.027115+0000 mgr.y (mgr.24512) 459 : cluster [DBG] pgmap v695: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 474 B/s wr, 2 op/s 2024-09-10T18:45:58.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:45:58 smithi151 bash[22565]: cluster 2024-09-10T18:45:58.027965+0000 mgr.y (mgr.24512) 460 : cluster [DBG] pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:45:58.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:45:58 smithi100 bash[18597]: cluster 2024-09-10T18:45:58.027965+0000 mgr.y (mgr.24512) 460 : cluster [DBG] pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:45:58.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:45:58 smithi100 bash[31607]: cluster 2024-09-10T18:45:58.027965+0000 mgr.y (mgr.24512) 460 : cluster [DBG] pgmap v696: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:46:00.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:00 smithi151 bash[22565]: cluster 2024-09-10T18:46:00.028783+0000 mgr.y (mgr.24512) 461 : cluster [DBG] pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:46:00.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:00 smithi100 bash[18597]: cluster 2024-09-10T18:46:00.028783+0000 mgr.y (mgr.24512) 461 : cluster [DBG] pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:46:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:00 smithi100 bash[31607]: cluster 2024-09-10T18:46:00.028783+0000 mgr.y (mgr.24512) 461 : cluster [DBG] pgmap v697: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:46:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:46:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:01] "GET /metrics HTTP/1.1" 200 228640 "" "Prometheus/2.43.0" 2024-09-10T18:46:02.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:02 smithi151 bash[22565]: cluster 2024-09-10T18:46:02.031000+0000 mgr.y (mgr.24512) 462 : cluster [DBG] pgmap v698: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:02.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:02 smithi100 bash[18597]: cluster 2024-09-10T18:46:02.031000+0000 mgr.y (mgr.24512) 462 : cluster [DBG] pgmap v698: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:02 smithi100 bash[31607]: cluster 2024-09-10T18:46:02.031000+0000 mgr.y (mgr.24512) 462 : cluster [DBG] pgmap v698: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:02.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:46:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:46:03.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:03 smithi151 bash[22565]: audit 2024-09-10T18:46:02.520014+0000 mgr.y (mgr.24512) 463 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:03 smithi100 bash[18597]: audit 2024-09-10T18:46:02.520014+0000 mgr.y (mgr.24512) 463 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:03 smithi100 bash[31607]: audit 2024-09-10T18:46:02.520014+0000 mgr.y (mgr.24512) 463 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:04.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:04 smithi151 bash[22565]: cluster 2024-09-10T18:46:04.031822+0000 mgr.y (mgr.24512) 464 : cluster [DBG] pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:46:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:04 smithi100 bash[18597]: cluster 2024-09-10T18:46:04.031822+0000 mgr.y (mgr.24512) 464 : cluster [DBG] pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:46:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:04 smithi100 bash[31607]: cluster 2024-09-10T18:46:04.031822+0000 mgr.y (mgr.24512) 464 : cluster [DBG] pgmap v699: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:46:04.923 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:46:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:05.673 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:46:05 smithi151 bash[26435]: debug 2024-09-10T18:46:05.283+0000 7fc3845d4640 -1 snap_mapper.add_oid found existing snaps mapped on 223:7cb13c74:test-rados-api-smithi100-72764-80::foo:2, removing 2024-09-10T18:46:05.674 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:46:05 smithi151 bash[47658]: debug 2024-09-10T18:46:05.283+0000 7f33a764e640 -1 snap_mapper.add_oid found existing snaps mapped on 223:7cb13c74:test-rados-api-smithi100-72764-80::foo:2, removing 2024-09-10T18:46:05.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:05 smithi151 bash[22565]: audit 2024-09-10T18:46:05.292399+0000 mon.b (mon.1) 313 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:05.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:05 smithi151 bash[22565]: audit 2024-09-10T18:46:05.294533+0000 mon.b (mon.1) 314 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-79"}]: dispatch 2024-09-10T18:46:05.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:05 smithi151 bash[22565]: audit 2024-09-10T18:46:05.304192+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:05.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:05 smithi151 bash[22565]: audit 2024-09-10T18:46:05.306099+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-79"}]: dispatch 2024-09-10T18:46:05.793 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[34936]: debug 2024-09-10T18:46:05.296+0000 7fc4c5350640 -1 snap_mapper.add_oid found existing snaps mapped on 223:7cb13c74:test-rados-api-smithi100-72764-80::foo:2, removing 2024-09-10T18:46:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[18597]: audit 2024-09-10T18:46:05.292399+0000 mon.b (mon.1) 313 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[18597]: audit 2024-09-10T18:46:05.294533+0000 mon.b (mon.1) 314 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-79"}]: dispatch 2024-09-10T18:46:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[18597]: audit 2024-09-10T18:46:05.304192+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[18597]: audit 2024-09-10T18:46:05.306099+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-79"}]: dispatch 2024-09-10T18:46:05.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[31607]: audit 2024-09-10T18:46:05.292399+0000 mon.b (mon.1) 313 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:05.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[31607]: audit 2024-09-10T18:46:05.294533+0000 mon.b (mon.1) 314 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-79"}]: dispatch 2024-09-10T18:46:05.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[31607]: audit 2024-09-10T18:46:05.304192+0000 mon.a (mon.0) 2644 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:05.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:05 smithi100 bash[31607]: audit 2024-09-10T18:46:05.306099+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-79"}]: dispatch 2024-09-10T18:46:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:06 smithi151 bash[22565]: cluster 2024-09-10T18:46:06.034348+0000 mgr.y (mgr.24512) 465 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:46:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:06 smithi100 bash[18597]: cluster 2024-09-10T18:46:06.034348+0000 mgr.y (mgr.24512) 465 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:46:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:06 smithi100 bash[31607]: cluster 2024-09-10T18:46:06.034348+0000 mgr.y (mgr.24512) 465 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:46:07.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:07 smithi151 bash[22565]: cluster 2024-09-10T18:46:06.366530+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-09-10T18:46:07.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:07 smithi100 bash[18597]: cluster 2024-09-10T18:46:06.366530+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-09-10T18:46:07.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:07 smithi100 bash[31607]: cluster 2024-09-10T18:46:06.366530+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-09-10T18:46:08.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:08 smithi151 bash[22565]: audit 2024-09-10T18:46:07.385796+0000 mon.b (mon.1) 315 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:08.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:08 smithi151 bash[22565]: cluster 2024-09-10T18:46:07.390972+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-09-10T18:46:08.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:08 smithi151 bash[22565]: audit 2024-09-10T18:46:07.397535+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:08.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:08 smithi151 bash[22565]: cluster 2024-09-10T18:46:08.035303+0000 mgr.y (mgr.24512) 466 : cluster [DBG] pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:46:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[18597]: audit 2024-09-10T18:46:07.385796+0000 mon.b (mon.1) 315 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[18597]: cluster 2024-09-10T18:46:07.390972+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-09-10T18:46:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[18597]: audit 2024-09-10T18:46:07.397535+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[18597]: cluster 2024-09-10T18:46:08.035303+0000 mgr.y (mgr.24512) 466 : cluster [DBG] pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:46:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[31607]: audit 2024-09-10T18:46:07.385796+0000 mon.b (mon.1) 315 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[31607]: cluster 2024-09-10T18:46:07.390972+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-09-10T18:46:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[31607]: audit 2024-09-10T18:46:07.397535+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:08.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:08 smithi100 bash[31607]: cluster 2024-09-10T18:46:08.035303+0000 mgr.y (mgr.24512) 466 : cluster [DBG] pgmap v703: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:46:09.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:09 smithi151 bash[22565]: cluster 2024-09-10T18:46:08.387011+0000 mon.a (mon.0) 2649 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:09.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:09 smithi151 bash[22565]: audit 2024-09-10T18:46:08.390669+0000 mon.b (mon.1) 316 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:09.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:09 smithi151 bash[22565]: audit 2024-09-10T18:46:08.394338+0000 mon.b (mon.1) 317 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:09.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:09 smithi151 bash[22565]: audit 2024-09-10T18:46:08.397220+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:09.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:09 smithi151 bash[22565]: cluster 2024-09-10T18:46:08.397693+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-09-10T18:46:09.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:09 smithi151 bash[22565]: audit 2024-09-10T18:46:08.412773+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:09.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:09 smithi151 bash[22565]: audit 2024-09-10T18:46:09.334143+0000 mon.c (mon.2) 461 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[18597]: cluster 2024-09-10T18:46:08.387011+0000 mon.a (mon.0) 2649 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[18597]: audit 2024-09-10T18:46:08.390669+0000 mon.b (mon.1) 316 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[18597]: audit 2024-09-10T18:46:08.394338+0000 mon.b (mon.1) 317 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[18597]: audit 2024-09-10T18:46:08.397220+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[18597]: cluster 2024-09-10T18:46:08.397693+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[18597]: audit 2024-09-10T18:46:08.412773+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[18597]: audit 2024-09-10T18:46:09.334143+0000 mon.c (mon.2) 461 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[31607]: cluster 2024-09-10T18:46:08.387011+0000 mon.a (mon.0) 2649 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[31607]: audit 2024-09-10T18:46:08.390669+0000 mon.b (mon.1) 316 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[31607]: audit 2024-09-10T18:46:08.394338+0000 mon.b (mon.1) 317 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[31607]: audit 2024-09-10T18:46:08.397220+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[31607]: cluster 2024-09-10T18:46:08.397693+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-09-10T18:46:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[31607]: audit 2024-09-10T18:46:08.412773+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:09 smithi100 bash[31607]: audit 2024-09-10T18:46:09.334143+0000 mon.c (mon.2) 461 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[18597]: audit 2024-09-10T18:46:09.411169+0000 mon.b (mon.1) 318 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[18597]: audit 2024-09-10T18:46:09.419952+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[18597]: cluster 2024-09-10T18:46:09.420065+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[18597]: audit 2024-09-10T18:46:09.424174+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[18597]: cluster 2024-09-10T18:46:10.036156+0000 mgr.y (mgr.24512) 467 : cluster [DBG] pgmap v706: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[31607]: audit 2024-09-10T18:46:09.411169+0000 mon.b (mon.1) 318 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[31607]: audit 2024-09-10T18:46:09.419952+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[31607]: cluster 2024-09-10T18:46:09.420065+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-09-10T18:46:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[31607]: audit 2024-09-10T18:46:09.424174+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:10 smithi100 bash[31607]: cluster 2024-09-10T18:46:10.036156+0000 mgr.y (mgr.24512) 467 : cluster [DBG] pgmap v706: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:10.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:10 smithi151 bash[22565]: audit 2024-09-10T18:46:09.411169+0000 mon.b (mon.1) 318 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:10.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:10 smithi151 bash[22565]: audit 2024-09-10T18:46:09.419952+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:10.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:10 smithi151 bash[22565]: cluster 2024-09-10T18:46:09.420065+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-09-10T18:46:10.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:10 smithi151 bash[22565]: audit 2024-09-10T18:46:09.424174+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:10.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:10 smithi151 bash[22565]: cluster 2024-09-10T18:46:10.036156+0000 mgr.y (mgr.24512) 467 : cluster [DBG] pgmap v706: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:11] "GET /metrics HTTP/1.1" 200 228562 "" "Prometheus/2.43.0" 2024-09-10T18:46:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: audit 2024-09-10T18:46:10.419420+0000 mon.b (mon.1) 319 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: audit 2024-09-10T18:46:10.428861+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: cluster 2024-09-10T18:46:10.429087+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-09-10T18:46:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: audit 2024-09-10T18:46:10.432188+0000 mon.a (mon.0) 2658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: audit 2024-09-10T18:46:11.423385+0000 mon.b (mon.1) 320 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: audit 2024-09-10T18:46:11.431969+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: cluster 2024-09-10T18:46:11.432101+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[18597]: audit 2024-09-10T18:46:11.434981+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: audit 2024-09-10T18:46:10.419420+0000 mon.b (mon.1) 319 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: audit 2024-09-10T18:46:10.428861+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: cluster 2024-09-10T18:46:10.429087+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: audit 2024-09-10T18:46:10.432188+0000 mon.a (mon.0) 2658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: audit 2024-09-10T18:46:11.423385+0000 mon.b (mon.1) 320 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: audit 2024-09-10T18:46:11.431969+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: cluster 2024-09-10T18:46:11.432101+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-09-10T18:46:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:11 smithi100 bash[31607]: audit 2024-09-10T18:46:11.434981+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:11.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: audit 2024-09-10T18:46:10.419420+0000 mon.b (mon.1) 319 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:11.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: audit 2024-09-10T18:46:10.428861+0000 mon.a (mon.0) 2656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:11.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: cluster 2024-09-10T18:46:10.429087+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-09-10T18:46:11.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: audit 2024-09-10T18:46:10.432188+0000 mon.a (mon.0) 2658 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:11.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: audit 2024-09-10T18:46:11.423385+0000 mon.b (mon.1) 320 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:11.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: audit 2024-09-10T18:46:11.431969+0000 mon.a (mon.0) 2659 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:11.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: cluster 2024-09-10T18:46:11.432101+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-09-10T18:46:11.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:11 smithi151 bash[22565]: audit 2024-09-10T18:46:11.434981+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:12 smithi100 bash[18597]: cluster 2024-09-10T18:46:12.037031+0000 mgr.y (mgr.24512) 468 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:12 smithi100 bash[18597]: audit 2024-09-10T18:46:12.428992+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:12 smithi100 bash[18597]: cluster 2024-09-10T18:46:12.429135+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-09-10T18:46:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:12 smithi100 bash[31607]: cluster 2024-09-10T18:46:12.037031+0000 mgr.y (mgr.24512) 468 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:12 smithi100 bash[31607]: audit 2024-09-10T18:46:12.428992+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:12 smithi100 bash[31607]: cluster 2024-09-10T18:46:12.429135+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-09-10T18:46:12.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:46:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:46:12.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:12 smithi151 bash[22565]: cluster 2024-09-10T18:46:12.037031+0000 mgr.y (mgr.24512) 468 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:12.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:12 smithi151 bash[22565]: audit 2024-09-10T18:46:12.428992+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:12.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:12 smithi151 bash[22565]: cluster 2024-09-10T18:46:12.429135+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-09-10T18:46:13.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:13 smithi100 bash[18597]: audit 2024-09-10T18:46:12.458667+0000 mon.b (mon.1) 321 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:46:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:13 smithi100 bash[18597]: audit 2024-09-10T18:46:12.470134+0000 mon.a (mon.0) 2664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:46:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:13 smithi100 bash[18597]: audit 2024-09-10T18:46:12.531228+0000 mgr.y (mgr.24512) 469 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:13 smithi100 bash[31607]: audit 2024-09-10T18:46:12.458667+0000 mon.b (mon.1) 321 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:46:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:13 smithi100 bash[31607]: audit 2024-09-10T18:46:12.470134+0000 mon.a (mon.0) 2664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:46:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:13 smithi100 bash[31607]: audit 2024-09-10T18:46:12.531228+0000 mgr.y (mgr.24512) 469 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:13.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:13 smithi151 bash[22565]: audit 2024-09-10T18:46:12.458667+0000 mon.b (mon.1) 321 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:46:13.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:13 smithi151 bash[22565]: audit 2024-09-10T18:46:12.470134+0000 mon.a (mon.0) 2664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:46:13.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:13 smithi151 bash[22565]: audit 2024-09-10T18:46:12.531228+0000 mgr.y (mgr.24512) 469 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[18597]: audit 2024-09-10T18:46:13.467701+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T18:46:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[18597]: cluster 2024-09-10T18:46:13.467833+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-09-10T18:46:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[18597]: audit 2024-09-10T18:46:13.477378+0000 mon.b (mon.1) 322 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:46:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[18597]: audit 2024-09-10T18:46:13.489125+0000 mon.a (mon.0) 2667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:46:14.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[18597]: cluster 2024-09-10T18:46:14.037890+0000 mgr.y (mgr.24512) 470 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:14.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[31607]: audit 2024-09-10T18:46:13.467701+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T18:46:14.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[31607]: cluster 2024-09-10T18:46:13.467833+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-09-10T18:46:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[31607]: audit 2024-09-10T18:46:13.477378+0000 mon.b (mon.1) 322 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:46:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[31607]: audit 2024-09-10T18:46:13.489125+0000 mon.a (mon.0) 2667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:46:14.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:14 smithi100 bash[31607]: cluster 2024-09-10T18:46:14.037890+0000 mgr.y (mgr.24512) 470 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:14.923 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:46:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:14.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:14 smithi151 bash[22565]: audit 2024-09-10T18:46:13.467701+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T18:46:14.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:14 smithi151 bash[22565]: cluster 2024-09-10T18:46:13.467833+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-09-10T18:46:14.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:14 smithi151 bash[22565]: audit 2024-09-10T18:46:13.477378+0000 mon.b (mon.1) 322 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:46:14.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:14 smithi151 bash[22565]: audit 2024-09-10T18:46:13.489125+0000 mon.a (mon.0) 2667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:46:14.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:14 smithi151 bash[22565]: cluster 2024-09-10T18:46:14.037890+0000 mgr.y (mgr.24512) 470 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[18597]: audit 2024-09-10T18:46:14.482148+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T18:46:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[18597]: cluster 2024-09-10T18:46:14.482316+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-09-10T18:46:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[18597]: audit 2024-09-10T18:46:14.484980+0000 mon.b (mon.1) 323 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[18597]: audit 2024-09-10T18:46:14.497250+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:15.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[31607]: audit 2024-09-10T18:46:14.482148+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T18:46:15.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[31607]: cluster 2024-09-10T18:46:14.482316+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-09-10T18:46:15.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[31607]: audit 2024-09-10T18:46:14.484980+0000 mon.b (mon.1) 323 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:15.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:15 smithi100 bash[31607]: audit 2024-09-10T18:46:14.497250+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:15.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:15 smithi151 bash[22565]: audit 2024-09-10T18:46:14.482148+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T18:46:15.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:15 smithi151 bash[22565]: cluster 2024-09-10T18:46:14.482316+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-09-10T18:46:15.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:15 smithi151 bash[22565]: audit 2024-09-10T18:46:14.484980+0000 mon.b (mon.1) 323 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:15.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:15 smithi151 bash[22565]: audit 2024-09-10T18:46:14.497250+0000 mon.a (mon.0) 2670 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: audit 2024-09-10T18:46:15.483111+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: cluster 2024-09-10T18:46:15.483396+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: audit 2024-09-10T18:46:15.521762+0000 mon.b (mon.1) 324 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: audit 2024-09-10T18:46:15.523649+0000 mon.b (mon.1) 325 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-81"}]: dispatch 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: audit 2024-09-10T18:46:15.533128+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: audit 2024-09-10T18:46:15.534888+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-81"}]: dispatch 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: cluster 2024-09-10T18:46:15.627614+0000 mon.a (mon.0) 2675 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[18597]: cluster 2024-09-10T18:46:16.038720+0000 mgr.y (mgr.24512) 471 : cluster [DBG] pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-09-10T18:46:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: audit 2024-09-10T18:46:15.483111+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: cluster 2024-09-10T18:46:15.483396+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-09-10T18:46:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: audit 2024-09-10T18:46:15.521762+0000 mon.b (mon.1) 324 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: audit 2024-09-10T18:46:15.523649+0000 mon.b (mon.1) 325 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-81"}]: dispatch 2024-09-10T18:46:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: audit 2024-09-10T18:46:15.533128+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: audit 2024-09-10T18:46:15.534888+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-81"}]: dispatch 2024-09-10T18:46:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: cluster 2024-09-10T18:46:15.627614+0000 mon.a (mon.0) 2675 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:16.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:16 smithi100 bash[31607]: cluster 2024-09-10T18:46:16.038720+0000 mgr.y (mgr.24512) 471 : cluster [DBG] pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-09-10T18:46:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: audit 2024-09-10T18:46:15.483111+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: cluster 2024-09-10T18:46:15.483396+0000 mon.a (mon.0) 2672 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-09-10T18:46:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: audit 2024-09-10T18:46:15.521762+0000 mon.b (mon.1) 324 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: audit 2024-09-10T18:46:15.523649+0000 mon.b (mon.1) 325 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-81"}]: dispatch 2024-09-10T18:46:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: audit 2024-09-10T18:46:15.533128+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: audit 2024-09-10T18:46:15.534888+0000 mon.a (mon.0) 2674 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-81"}]: dispatch 2024-09-10T18:46:16.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: cluster 2024-09-10T18:46:15.627614+0000 mon.a (mon.0) 2675 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:16.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:16 smithi151 bash[22565]: cluster 2024-09-10T18:46:16.038720+0000 mgr.y (mgr.24512) 471 : cluster [DBG] pgmap v715: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-09-10T18:46:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:17 smithi100 bash[18597]: cluster 2024-09-10T18:46:16.502324+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-09-10T18:46:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:17 smithi100 bash[31607]: cluster 2024-09-10T18:46:16.502324+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-09-10T18:46:17.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:17 smithi151 bash[22565]: cluster 2024-09-10T18:46:16.502324+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-09-10T18:46:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[18597]: cluster 2024-09-10T18:46:17.508537+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-09-10T18:46:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[18597]: audit 2024-09-10T18:46:17.510899+0000 mon.b (mon.1) 326 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[18597]: audit 2024-09-10T18:46:17.523127+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[18597]: cluster 2024-09-10T18:46:18.039550+0000 mgr.y (mgr.24512) 472 : cluster [DBG] pgmap v718: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-09-10T18:46:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[18597]: audit 2024-09-10T18:46:18.510786+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[18597]: cluster 2024-09-10T18:46:18.511190+0000 mon.a (mon.0) 2680 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-09-10T18:46:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[31607]: cluster 2024-09-10T18:46:17.508537+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-09-10T18:46:18.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[31607]: audit 2024-09-10T18:46:17.510899+0000 mon.b (mon.1) 326 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:18.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[31607]: audit 2024-09-10T18:46:17.523127+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:18.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[31607]: cluster 2024-09-10T18:46:18.039550+0000 mgr.y (mgr.24512) 472 : cluster [DBG] pgmap v718: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-09-10T18:46:18.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[31607]: audit 2024-09-10T18:46:18.510786+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:18.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:18 smithi100 bash[31607]: cluster 2024-09-10T18:46:18.511190+0000 mon.a (mon.0) 2680 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-09-10T18:46:18.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:18 smithi151 bash[22565]: cluster 2024-09-10T18:46:17.508537+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-09-10T18:46:18.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:18 smithi151 bash[22565]: audit 2024-09-10T18:46:17.510899+0000 mon.b (mon.1) 326 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:18.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:18 smithi151 bash[22565]: audit 2024-09-10T18:46:17.523127+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:18.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:18 smithi151 bash[22565]: cluster 2024-09-10T18:46:18.039550+0000 mgr.y (mgr.24512) 472 : cluster [DBG] pgmap v718: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-09-10T18:46:18.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:18 smithi151 bash[22565]: audit 2024-09-10T18:46:18.510786+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:18.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:18 smithi151 bash[22565]: cluster 2024-09-10T18:46:18.511190+0000 mon.a (mon.0) 2680 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-09-10T18:46:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:19 smithi100 bash[18597]: audit 2024-09-10T18:46:18.507972+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:19 smithi100 bash[18597]: audit 2024-09-10T18:46:18.520157+0000 mon.b (mon.1) 328 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:19 smithi100 bash[18597]: audit 2024-09-10T18:46:18.537018+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:19 smithi100 bash[31607]: audit 2024-09-10T18:46:18.507972+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:19.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:19 smithi100 bash[31607]: audit 2024-09-10T18:46:18.520157+0000 mon.b (mon.1) 328 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:19.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:19 smithi100 bash[31607]: audit 2024-09-10T18:46:18.537018+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:19.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:19 smithi151 bash[22565]: audit 2024-09-10T18:46:18.507972+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:19.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:19 smithi151 bash[22565]: audit 2024-09-10T18:46:18.520157+0000 mon.b (mon.1) 328 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:19.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:19 smithi151 bash[22565]: audit 2024-09-10T18:46:18.537018+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:20.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[18597]: audit 2024-09-10T18:46:19.517571+0000 mon.b (mon.1) 329 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[18597]: audit 2024-09-10T18:46:19.524568+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[18597]: cluster 2024-09-10T18:46:19.524787+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[18597]: audit 2024-09-10T18:46:19.536282+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[18597]: cluster 2024-09-10T18:46:20.040354+0000 mgr.y (mgr.24512) 473 : cluster [DBG] pgmap v721: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[31607]: audit 2024-09-10T18:46:19.517571+0000 mon.b (mon.1) 329 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[31607]: audit 2024-09-10T18:46:19.524568+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[31607]: cluster 2024-09-10T18:46:19.524787+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[31607]: audit 2024-09-10T18:46:19.536282+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:20.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:20 smithi100 bash[31607]: cluster 2024-09-10T18:46:20.040354+0000 mgr.y (mgr.24512) 473 : cluster [DBG] pgmap v721: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:20.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:20 smithi151 bash[22565]: audit 2024-09-10T18:46:19.517571+0000 mon.b (mon.1) 329 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:20.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:20 smithi151 bash[22565]: audit 2024-09-10T18:46:19.524568+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:20.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:20 smithi151 bash[22565]: cluster 2024-09-10T18:46:19.524787+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-09-10T18:46:20.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:20 smithi151 bash[22565]: audit 2024-09-10T18:46:19.536282+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:20.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:20 smithi151 bash[22565]: cluster 2024-09-10T18:46:20.040354+0000 mgr.y (mgr.24512) 473 : cluster [DBG] pgmap v721: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:21] "GET /metrics HTTP/1.1" 200 228562 "" "Prometheus/2.43.0" 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[18597]: audit 2024-09-10T18:46:20.541460+0000 mon.b (mon.1) 330 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[18597]: audit 2024-09-10T18:46:20.547929+0000 mon.a (mon.0) 2685 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[18597]: cluster 2024-09-10T18:46:20.548029+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[18597]: audit 2024-09-10T18:46:20.552704+0000 mon.a (mon.0) 2687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[31607]: audit 2024-09-10T18:46:20.541460+0000 mon.b (mon.1) 330 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[31607]: audit 2024-09-10T18:46:20.547929+0000 mon.a (mon.0) 2685 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[31607]: cluster 2024-09-10T18:46:20.548029+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-09-10T18:46:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:21 smithi100 bash[31607]: audit 2024-09-10T18:46:20.552704+0000 mon.a (mon.0) 2687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:21.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:21 smithi151 bash[22565]: audit 2024-09-10T18:46:20.541460+0000 mon.b (mon.1) 330 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:21.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:21 smithi151 bash[22565]: audit 2024-09-10T18:46:20.547929+0000 mon.a (mon.0) 2685 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:21.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:21 smithi151 bash[22565]: cluster 2024-09-10T18:46:20.548029+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-09-10T18:46:21.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:21 smithi151 bash[22565]: audit 2024-09-10T18:46:20.552704+0000 mon.a (mon.0) 2687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:22.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:46:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:46:22.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:22 smithi151 bash[22565]: audit 2024-09-10T18:46:21.548907+0000 mon.b (mon.1) 331 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:22.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:22 smithi151 bash[22565]: audit 2024-09-10T18:46:21.555146+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:22.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:22 smithi151 bash[22565]: cluster 2024-09-10T18:46:21.555361+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-09-10T18:46:22.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:22 smithi151 bash[22565]: audit 2024-09-10T18:46:21.565766+0000 mon.a (mon.0) 2690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:22.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:22 smithi151 bash[22565]: cluster 2024-09-10T18:46:22.041200+0000 mgr.y (mgr.24512) 474 : cluster [DBG] pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T18:46:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[18597]: audit 2024-09-10T18:46:21.548907+0000 mon.b (mon.1) 331 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[18597]: audit 2024-09-10T18:46:21.555146+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[18597]: cluster 2024-09-10T18:46:21.555361+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-09-10T18:46:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[18597]: audit 2024-09-10T18:46:21.565766+0000 mon.a (mon.0) 2690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[18597]: cluster 2024-09-10T18:46:22.041200+0000 mgr.y (mgr.24512) 474 : cluster [DBG] pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T18:46:23.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[31607]: audit 2024-09-10T18:46:21.548907+0000 mon.b (mon.1) 331 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:23.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[31607]: audit 2024-09-10T18:46:21.555146+0000 mon.a (mon.0) 2688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:23.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[31607]: cluster 2024-09-10T18:46:21.555361+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-09-10T18:46:23.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[31607]: audit 2024-09-10T18:46:21.565766+0000 mon.a (mon.0) 2690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:23.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:22 smithi100 bash[31607]: cluster 2024-09-10T18:46:22.041200+0000 mgr.y (mgr.24512) 474 : cluster [DBG] pgmap v724: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T18:46:23.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:23 smithi151 bash[22565]: audit 2024-09-10T18:46:22.542412+0000 mgr.y (mgr.24512) 475 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:23.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:23 smithi151 bash[22565]: audit 2024-09-10T18:46:22.570852+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:23.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:23 smithi151 bash[22565]: cluster 2024-09-10T18:46:22.571360+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-09-10T18:46:24.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:23 smithi100 bash[31607]: audit 2024-09-10T18:46:22.542412+0000 mgr.y (mgr.24512) 475 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:24.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:23 smithi100 bash[31607]: audit 2024-09-10T18:46:22.570852+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:24.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:23 smithi100 bash[31607]: cluster 2024-09-10T18:46:22.571360+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-09-10T18:46:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:23 smithi100 bash[18597]: audit 2024-09-10T18:46:22.542412+0000 mgr.y (mgr.24512) 475 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:23 smithi100 bash[18597]: audit 2024-09-10T18:46:22.570852+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:23 smithi100 bash[18597]: cluster 2024-09-10T18:46:22.571360+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-09-10T18:46:24.923 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:46:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:24.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:24 smithi151 bash[22565]: cluster 2024-09-10T18:46:23.583803+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-09-10T18:46:24.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:24 smithi151 bash[22565]: cluster 2024-09-10T18:46:24.042111+0000 mgr.y (mgr.24512) 476 : cluster [DBG] pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T18:46:24.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:24 smithi151 bash[22565]: audit 2024-09-10T18:46:24.334461+0000 mon.c (mon.2) 462 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:25.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:24 smithi100 bash[18597]: cluster 2024-09-10T18:46:23.583803+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-09-10T18:46:25.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:24 smithi100 bash[18597]: cluster 2024-09-10T18:46:24.042111+0000 mgr.y (mgr.24512) 476 : cluster [DBG] pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T18:46:25.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:24 smithi100 bash[18597]: audit 2024-09-10T18:46:24.334461+0000 mon.c (mon.2) 462 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:24 smithi100 bash[31607]: cluster 2024-09-10T18:46:23.583803+0000 mon.a (mon.0) 2693 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-09-10T18:46:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:24 smithi100 bash[31607]: cluster 2024-09-10T18:46:24.042111+0000 mgr.y (mgr.24512) 476 : cluster [DBG] pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T18:46:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:24 smithi100 bash[31607]: audit 2024-09-10T18:46:24.334461+0000 mon.c (mon.2) 462 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:25.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:25 smithi151 bash[22565]: cluster 2024-09-10T18:46:24.597665+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-09-10T18:46:25.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:25 smithi151 bash[22565]: audit 2024-09-10T18:46:24.626291+0000 mon.b (mon.1) 332 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:25.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:25 smithi151 bash[22565]: audit 2024-09-10T18:46:24.627701+0000 mon.b (mon.1) 333 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-83"}]: dispatch 2024-09-10T18:46:25.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:25 smithi151 bash[22565]: audit 2024-09-10T18:46:24.637358+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:25.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:25 smithi151 bash[22565]: audit 2024-09-10T18:46:24.638644+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-83"}]: dispatch 2024-09-10T18:46:26.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[31607]: cluster 2024-09-10T18:46:24.597665+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-09-10T18:46:26.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[31607]: audit 2024-09-10T18:46:24.626291+0000 mon.b (mon.1) 332 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[31607]: audit 2024-09-10T18:46:24.627701+0000 mon.b (mon.1) 333 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-83"}]: dispatch 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[31607]: audit 2024-09-10T18:46:24.637358+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[31607]: audit 2024-09-10T18:46:24.638644+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-83"}]: dispatch 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[18597]: cluster 2024-09-10T18:46:24.597665+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[18597]: audit 2024-09-10T18:46:24.626291+0000 mon.b (mon.1) 332 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[18597]: audit 2024-09-10T18:46:24.627701+0000 mon.b (mon.1) 333 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-83"}]: dispatch 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[18597]: audit 2024-09-10T18:46:24.637358+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:25 smithi100 bash[18597]: audit 2024-09-10T18:46:24.638644+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-83"}]: dispatch 2024-09-10T18:46:26.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:26 smithi151 bash[22565]: cluster 2024-09-10T18:46:25.615689+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-09-10T18:46:26.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:26 smithi151 bash[22565]: cluster 2024-09-10T18:46:26.042890+0000 mgr.y (mgr.24512) 477 : cluster [DBG] pgmap v730: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:46:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:26 smithi100 bash[18597]: cluster 2024-09-10T18:46:25.615689+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-09-10T18:46:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:26 smithi100 bash[18597]: cluster 2024-09-10T18:46:26.042890+0000 mgr.y (mgr.24512) 477 : cluster [DBG] pgmap v730: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:46:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:26 smithi100 bash[31607]: cluster 2024-09-10T18:46:25.615689+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-09-10T18:46:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:26 smithi100 bash[31607]: cluster 2024-09-10T18:46:26.042890+0000 mgr.y (mgr.24512) 477 : cluster [DBG] pgmap v730: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:46:27.673 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:46:27 smithi151 bash[57256]: logger=cleanup t=2024-09-10T18:46:27.378900651Z level=info msg="Completed cleanup jobs" duration=80.890362ms 2024-09-10T18:46:27.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:27 smithi151 bash[22565]: audit 2024-09-10T18:46:26.627777+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:27.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:27 smithi151 bash[22565]: cluster 2024-09-10T18:46:26.628028+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-09-10T18:46:27.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:27 smithi151 bash[22565]: audit 2024-09-10T18:46:26.638729+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:27 smithi100 bash[18597]: audit 2024-09-10T18:46:26.627777+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:27 smithi100 bash[18597]: cluster 2024-09-10T18:46:26.628028+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-09-10T18:46:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:27 smithi100 bash[18597]: audit 2024-09-10T18:46:26.638729+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:27 smithi100 bash[31607]: audit 2024-09-10T18:46:26.627777+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:27 smithi100 bash[31607]: cluster 2024-09-10T18:46:26.628028+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-09-10T18:46:28.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:27 smithi100 bash[31607]: audit 2024-09-10T18:46:26.638729+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:28.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:28 smithi151 bash[22565]: audit 2024-09-10T18:46:27.625370+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:28.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:28 smithi151 bash[22565]: audit 2024-09-10T18:46:27.633062+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:28.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:28 smithi151 bash[22565]: cluster 2024-09-10T18:46:27.633371+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-09-10T18:46:28.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:28 smithi151 bash[22565]: audit 2024-09-10T18:46:27.639624+0000 mon.b (mon.1) 336 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:28.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:28 smithi151 bash[22565]: audit 2024-09-10T18:46:27.650611+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:28.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:28 smithi151 bash[22565]: cluster 2024-09-10T18:46:28.043798+0000 mgr.y (mgr.24512) 478 : cluster [DBG] pgmap v733: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:46:29.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[18597]: audit 2024-09-10T18:46:27.625370+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[18597]: audit 2024-09-10T18:46:27.633062+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[18597]: cluster 2024-09-10T18:46:27.633371+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[18597]: audit 2024-09-10T18:46:27.639624+0000 mon.b (mon.1) 336 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[18597]: audit 2024-09-10T18:46:27.650611+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[18597]: cluster 2024-09-10T18:46:28.043798+0000 mgr.y (mgr.24512) 478 : cluster [DBG] pgmap v733: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[31607]: audit 2024-09-10T18:46:27.625370+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[31607]: audit 2024-09-10T18:46:27.633062+0000 mon.a (mon.0) 2700 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[31607]: cluster 2024-09-10T18:46:27.633371+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-09-10T18:46:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[31607]: audit 2024-09-10T18:46:27.639624+0000 mon.b (mon.1) 336 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:29.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[31607]: audit 2024-09-10T18:46:27.650611+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:29.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:28 smithi100 bash[31607]: cluster 2024-09-10T18:46:28.043798+0000 mgr.y (mgr.24512) 478 : cluster [DBG] pgmap v733: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:46:29.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:29 smithi151 bash[22565]: audit 2024-09-10T18:46:28.647741+0000 mon.b (mon.1) 337 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:29.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:29 smithi151 bash[22565]: audit 2024-09-10T18:46:28.656814+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:29.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:29 smithi151 bash[22565]: cluster 2024-09-10T18:46:28.656996+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-09-10T18:46:29.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:29 smithi151 bash[22565]: audit 2024-09-10T18:46:28.660669+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:30.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[31607]: audit 2024-09-10T18:46:28.647741+0000 mon.b (mon.1) 337 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:30.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[31607]: audit 2024-09-10T18:46:28.656814+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:30.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[31607]: cluster 2024-09-10T18:46:28.656996+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-09-10T18:46:30.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[31607]: audit 2024-09-10T18:46:28.660669+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[18597]: audit 2024-09-10T18:46:28.647741+0000 mon.b (mon.1) 337 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[18597]: audit 2024-09-10T18:46:28.656814+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[18597]: cluster 2024-09-10T18:46:28.656996+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-09-10T18:46:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:29 smithi100 bash[18597]: audit 2024-09-10T18:46:28.660669+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[18597]: audit 2024-09-10T18:46:29.659604+0000 mon.b (mon.1) 338 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[18597]: audit 2024-09-10T18:46:29.668114+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[18597]: cluster 2024-09-10T18:46:29.668225+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[18597]: audit 2024-09-10T18:46:29.671628+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[18597]: cluster 2024-09-10T18:46:30.044846+0000 mgr.y (mgr.24512) 479 : cluster [DBG] pgmap v736: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[31607]: audit 2024-09-10T18:46:29.659604+0000 mon.b (mon.1) 338 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[31607]: audit 2024-09-10T18:46:29.668114+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[31607]: cluster 2024-09-10T18:46:29.668225+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-09-10T18:46:31.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[31607]: audit 2024-09-10T18:46:29.671628+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:30 smithi100 bash[31607]: cluster 2024-09-10T18:46:30.044846+0000 mgr.y (mgr.24512) 479 : cluster [DBG] pgmap v736: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:31.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:30 smithi151 bash[22565]: audit 2024-09-10T18:46:29.659604+0000 mon.b (mon.1) 338 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:31.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:30 smithi151 bash[22565]: audit 2024-09-10T18:46:29.668114+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_tier","val": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:46:31.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:30 smithi151 bash[22565]: cluster 2024-09-10T18:46:29.668225+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-09-10T18:46:31.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:30 smithi151 bash[22565]: audit 2024-09-10T18:46:29.671628+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:31.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:30 smithi151 bash[22565]: cluster 2024-09-10T18:46:30.044846+0000 mgr.y (mgr.24512) 479 : cluster [DBG] pgmap v736: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:31.688 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:31] "GET /metrics HTTP/1.1" 200 228613 "" "Prometheus/2.43.0" 2024-09-10T18:46:31.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[18597]: audit 2024-09-10T18:46:30.670486+0000 mon.b (mon.1) 339 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:31.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[18597]: audit 2024-09-10T18:46:30.677548+0000 mon.a (mon.0) 2709 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:31.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[18597]: cluster 2024-09-10T18:46:30.677697+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-09-10T18:46:31.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[18597]: audit 2024-09-10T18:46:30.681352+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:31.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[31607]: audit 2024-09-10T18:46:30.670486+0000 mon.b (mon.1) 339 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:31.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[31607]: audit 2024-09-10T18:46:30.677548+0000 mon.a (mon.0) 2709 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:31.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[31607]: cluster 2024-09-10T18:46:30.677697+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-09-10T18:46:31.689 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:31 smithi100 bash[31607]: audit 2024-09-10T18:46:30.681352+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:32.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:31 smithi151 bash[22565]: audit 2024-09-10T18:46:30.670486+0000 mon.b (mon.1) 339 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:32.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:31 smithi151 bash[22565]: audit 2024-09-10T18:46:30.677548+0000 mon.a (mon.0) 2709 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:32.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:31 smithi151 bash[22565]: cluster 2024-09-10T18:46:30.677697+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-09-10T18:46:32.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:31 smithi151 bash[22565]: audit 2024-09-10T18:46:30.681352+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:32.922 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:46:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:46:32.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:32 smithi151 bash[22565]: audit 2024-09-10T18:46:31.682629+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:32.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:32 smithi151 bash[22565]: cluster 2024-09-10T18:46:31.683039+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-09-10T18:46:32.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:32 smithi151 bash[22565]: cluster 2024-09-10T18:46:32.045730+0000 mgr.y (mgr.24512) 480 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:32 smithi100 bash[18597]: audit 2024-09-10T18:46:31.682629+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:32 smithi100 bash[18597]: cluster 2024-09-10T18:46:31.683039+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-09-10T18:46:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:32 smithi100 bash[18597]: cluster 2024-09-10T18:46:32.045730+0000 mgr.y (mgr.24512) 480 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:32 smithi100 bash[31607]: audit 2024-09-10T18:46:31.682629+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:32 smithi100 bash[31607]: cluster 2024-09-10T18:46:31.683039+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-09-10T18:46:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:32 smithi100 bash[31607]: cluster 2024-09-10T18:46:32.045730+0000 mgr.y (mgr.24512) 480 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:34.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:33 smithi100 bash[18597]: audit 2024-09-10T18:46:32.553608+0000 mgr.y (mgr.24512) 481 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:34.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:33 smithi100 bash[18597]: cluster 2024-09-10T18:46:32.700818+0000 mon.a (mon.0) 2714 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-09-10T18:46:34.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:33 smithi100 bash[31607]: audit 2024-09-10T18:46:32.553608+0000 mgr.y (mgr.24512) 481 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:34.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:33 smithi100 bash[31607]: cluster 2024-09-10T18:46:32.700818+0000 mon.a (mon.0) 2714 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-09-10T18:46:34.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:33 smithi151 bash[22565]: audit 2024-09-10T18:46:32.553608+0000 mgr.y (mgr.24512) 481 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:34.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:33 smithi151 bash[22565]: cluster 2024-09-10T18:46:32.700818+0000 mon.a (mon.0) 2714 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-09-10T18:46:34.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:46:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:34 smithi151 bash[22565]: cluster 2024-09-10T18:46:33.714119+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-09-10T18:46:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:34 smithi151 bash[22565]: audit 2024-09-10T18:46:33.746678+0000 mon.b (mon.1) 340 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:34 smithi151 bash[22565]: audit 2024-09-10T18:46:33.748131+0000 mon.b (mon.1) 341 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-85"}]: dispatch 2024-09-10T18:46:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:34 smithi151 bash[22565]: audit 2024-09-10T18:46:33.757565+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:34 smithi151 bash[22565]: audit 2024-09-10T18:46:33.758836+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-85"}]: dispatch 2024-09-10T18:46:34.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:34 smithi151 bash[22565]: cluster 2024-09-10T18:46:34.046496+0000 mgr.y (mgr.24512) 482 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[18597]: cluster 2024-09-10T18:46:33.714119+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[18597]: audit 2024-09-10T18:46:33.746678+0000 mon.b (mon.1) 340 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[18597]: audit 2024-09-10T18:46:33.748131+0000 mon.b (mon.1) 341 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-85"}]: dispatch 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[18597]: audit 2024-09-10T18:46:33.757565+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[18597]: audit 2024-09-10T18:46:33.758836+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-85"}]: dispatch 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[18597]: cluster 2024-09-10T18:46:34.046496+0000 mgr.y (mgr.24512) 482 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[31607]: cluster 2024-09-10T18:46:33.714119+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-09-10T18:46:35.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[31607]: audit 2024-09-10T18:46:33.746678+0000 mon.b (mon.1) 340 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:35.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[31607]: audit 2024-09-10T18:46:33.748131+0000 mon.b (mon.1) 341 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-85"}]: dispatch 2024-09-10T18:46:35.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[31607]: audit 2024-09-10T18:46:33.757565+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:35.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[31607]: audit 2024-09-10T18:46:33.758836+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-85"}]: dispatch 2024-09-10T18:46:35.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:34 smithi100 bash[31607]: cluster 2024-09-10T18:46:34.046496+0000 mgr.y (mgr.24512) 482 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:36.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:35 smithi100 bash[18597]: cluster 2024-09-10T18:46:34.743010+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-09-10T18:46:36.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:35 smithi100 bash[31607]: cluster 2024-09-10T18:46:34.743010+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-09-10T18:46:36.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:35 smithi151 bash[22565]: cluster 2024-09-10T18:46:34.743010+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-09-10T18:46:36.773 INFO:tasks.workunit.client.0.smithi100.stdout:K ] LibRadosTwoPoolsPP.ProxyRead (18153 ms) 2024-09-10T18:46:36.773 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-10T18:46:36.773 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (23093 ms) 2024-09-10T18:46:36.773 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-10T18:46:36.773 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3039 ms) 2024-09-10T18:46:36.773 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3044 ms) 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3011 ms) 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3008 ms) 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4035 ms) 2024-09-10T18:46:36.774 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39531 ms) 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16367 ms) 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4060 ms) 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3022 ms) 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-10T18:46:36.775 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24913 ms) 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15013 ms) 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5055 ms) 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4052 ms) 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.776 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: waiting for scrubs... 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: done waiting 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25173 ms) 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10139 ms) 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9111 ms) 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9134 ms) 2024-09-10T18:46:36.777 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-10T18:46:37.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[18597]: audit 2024-09-10T18:46:35.746887+0000 mon.b (mon.1) 342 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:37.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[18597]: cluster 2024-09-10T18:46:35.749853+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-09-10T18:46:37.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[18597]: audit 2024-09-10T18:46:35.757838+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:37.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[18597]: cluster 2024-09-10T18:46:36.047352+0000 mgr.y (mgr.24512) 483 : cluster [DBG] pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:46:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[31607]: audit 2024-09-10T18:46:35.746887+0000 mon.b (mon.1) 342 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[31607]: cluster 2024-09-10T18:46:35.749853+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-09-10T18:46:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[31607]: audit 2024-09-10T18:46:35.757838+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:37.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:36 smithi100 bash[31607]: cluster 2024-09-10T18:46:36.047352+0000 mgr.y (mgr.24512) 483 : cluster [DBG] pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:46:37.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:36 smithi151 bash[22565]: audit 2024-09-10T18:46:35.746887+0000 mon.b (mon.1) 342 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:37.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:36 smithi151 bash[22565]: cluster 2024-09-10T18:46:35.749853+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-09-10T18:46:37.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:36 smithi151 bash[22565]: audit 2024-09-10T18:46:35.757838+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:37.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:36 smithi151 bash[22565]: cluster 2024-09-10T18:46:36.047352+0000 mgr.y (mgr.24512) 483 : cluster [DBG] pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:46:38.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[18597]: cluster 2024-09-10T18:46:36.745906+0000 mon.a (mon.0) 2721 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[18597]: audit 2024-09-10T18:46:36.757064+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[18597]: audit 2024-09-10T18:46:36.764895+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[18597]: cluster 2024-09-10T18:46:36.765082+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-09-10T18:46:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[31607]: cluster 2024-09-10T18:46:36.745906+0000 mon.a (mon.0) 2721 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[31607]: audit 2024-09-10T18:46:36.757064+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[31607]: audit 2024-09-10T18:46:36.764895+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:37 smithi100 bash[31607]: cluster 2024-09-10T18:46:36.765082+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-09-10T18:46:38.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:37 smithi151 bash[22565]: cluster 2024-09-10T18:46:36.745906+0000 mon.a (mon.0) 2721 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:38.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:37 smithi151 bash[22565]: audit 2024-09-10T18:46:36.757064+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:38.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:37 smithi151 bash[22565]: audit 2024-09-10T18:46:36.764895+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:38.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:37 smithi151 bash[22565]: cluster 2024-09-10T18:46:36.765082+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-09-10T18:46:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: audit 2024-09-10T18:46:37.760765+0000 mon.b (mon.1) 344 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: cluster 2024-09-10T18:46:37.768888+0000 mon.a (mon.0) 2724 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-09-10T18:46:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: audit 2024-09-10T18:46:37.777956+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: cluster 2024-09-10T18:46:38.048244+0000 mgr.y (mgr.24512) 484 : cluster [DBG] pgmap v748: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:46:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: audit 2024-09-10T18:46:38.762042+0000 mon.b (mon.1) 345 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]: dispatch 2024-09-10T18:46:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: audit 2024-09-10T18:46:38.769757+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: cluster 2024-09-10T18:46:38.770047+0000 mon.a (mon.0) 2727 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[18597]: audit 2024-09-10T18:46:38.773001+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]: dispatch 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: audit 2024-09-10T18:46:37.760765+0000 mon.b (mon.1) 344 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: cluster 2024-09-10T18:46:37.768888+0000 mon.a (mon.0) 2724 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: audit 2024-09-10T18:46:37.777956+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: cluster 2024-09-10T18:46:38.048244+0000 mgr.y (mgr.24512) 484 : cluster [DBG] pgmap v748: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: audit 2024-09-10T18:46:38.762042+0000 mon.b (mon.1) 345 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]: dispatch 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: audit 2024-09-10T18:46:38.769757+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:39.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: cluster 2024-09-10T18:46:38.770047+0000 mon.a (mon.0) 2727 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-09-10T18:46:39.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:38 smithi100 bash[31607]: audit 2024-09-10T18:46:38.773001+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]: dispatch 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: audit 2024-09-10T18:46:37.760765+0000 mon.b (mon.1) 344 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: cluster 2024-09-10T18:46:37.768888+0000 mon.a (mon.0) 2724 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: audit 2024-09-10T18:46:37.777956+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: cluster 2024-09-10T18:46:38.048244+0000 mgr.y (mgr.24512) 484 : cluster [DBG] pgmap v748: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: audit 2024-09-10T18:46:38.762042+0000 mon.b (mon.1) 345 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]: dispatch 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: audit 2024-09-10T18:46:38.769757+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: cluster 2024-09-10T18:46:38.770047+0000 mon.a (mon.0) 2727 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-09-10T18:46:39.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:38 smithi151 bash[22565]: audit 2024-09-10T18:46:38.773001+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]: dispatch 2024-09-10T18:46:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:39 smithi151 bash[22565]: audit 2024-09-10T18:46:39.335039+0000 mon.c (mon.2) 463 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:40.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:39 smithi151 bash[22565]: audit 2024-09-10T18:46:39.611942+0000 mon.c (mon.2) 464 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:46:40.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:39 smithi151 bash[22565]: audit 2024-09-10T18:46:39.765234+0000 mon.b (mon.1) 346 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:40.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:39 smithi151 bash[22565]: audit 2024-09-10T18:46:39.771955+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]': finished 2024-09-10T18:46:40.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:39 smithi151 bash[22565]: cluster 2024-09-10T18:46:39.772631+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-09-10T18:46:40.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:39 smithi151 bash[22565]: audit 2024-09-10T18:46:39.777027+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:40.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[18597]: audit 2024-09-10T18:46:39.335039+0000 mon.c (mon.2) 463 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[18597]: audit 2024-09-10T18:46:39.611942+0000 mon.c (mon.2) 464 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[18597]: audit 2024-09-10T18:46:39.765234+0000 mon.b (mon.1) 346 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[18597]: audit 2024-09-10T18:46:39.771955+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]': finished 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[18597]: cluster 2024-09-10T18:46:39.772631+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[18597]: audit 2024-09-10T18:46:39.777027+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[31607]: audit 2024-09-10T18:46:39.335039+0000 mon.c (mon.2) 463 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[31607]: audit 2024-09-10T18:46:39.611942+0000 mon.c (mon.2) 464 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[31607]: audit 2024-09-10T18:46:39.765234+0000 mon.b (mon.1) 346 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[31607]: audit 2024-09-10T18:46:39.771955+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_tier","val": "test-rados-api-smithi100-72764-89-test-flush"}]': finished 2024-09-10T18:46:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[31607]: cluster 2024-09-10T18:46:39.772631+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-09-10T18:46:40.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:39 smithi100 bash[31607]: audit 2024-09-10T18:46:39.777027+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:46:41.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:40 smithi151 bash[22565]: cluster 2024-09-10T18:46:40.049088+0000 mgr.y (mgr.24512) 485 : cluster [DBG] pgmap v751: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:41.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:40 smithi151 bash[22565]: audit 2024-09-10T18:46:40.769892+0000 mon.b (mon.1) 347 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:41.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:40 smithi151 bash[22565]: audit 2024-09-10T18:46:40.777230+0000 mon.a (mon.0) 2732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:41.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:40 smithi151 bash[22565]: cluster 2024-09-10T18:46:40.777445+0000 mon.a (mon.0) 2733 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-09-10T18:46:41.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:40 smithi151 bash[22565]: audit 2024-09-10T18:46:40.781152+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:41.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[31607]: cluster 2024-09-10T18:46:40.049088+0000 mgr.y (mgr.24512) 485 : cluster [DBG] pgmap v751: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[31607]: audit 2024-09-10T18:46:40.769892+0000 mon.b (mon.1) 347 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[31607]: audit 2024-09-10T18:46:40.777230+0000 mon.a (mon.0) 2732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[31607]: cluster 2024-09-10T18:46:40.777445+0000 mon.a (mon.0) 2733 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[31607]: audit 2024-09-10T18:46:40.781152+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[18597]: cluster 2024-09-10T18:46:40.049088+0000 mgr.y (mgr.24512) 485 : cluster [DBG] pgmap v751: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[18597]: audit 2024-09-10T18:46:40.769892+0000 mon.b (mon.1) 347 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[18597]: audit 2024-09-10T18:46:40.777230+0000 mon.a (mon.0) 2732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[18597]: cluster 2024-09-10T18:46:40.777445+0000 mon.a (mon.0) 2733 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-09-10T18:46:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:40 smithi100 bash[18597]: audit 2024-09-10T18:46:40.781152+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:46:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:46:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:41] "GET /metrics HTTP/1.1" 200 231064 "" "Prometheus/2.43.0" 2024-09-10T18:46:42.922 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:46:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:46:42.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:42 smithi151 bash[22565]: audit 2024-09-10T18:46:41.780994+0000 mon.a (mon.0) 2735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:42.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:42 smithi151 bash[22565]: cluster 2024-09-10T18:46:41.781123+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-09-10T18:46:42.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:42 smithi151 bash[22565]: cluster 2024-09-10T18:46:42.049965+0000 mgr.y (mgr.24512) 486 : cluster [DBG] pgmap v754: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:43.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:42 smithi100 bash[18597]: audit 2024-09-10T18:46:41.780994+0000 mon.a (mon.0) 2735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:42 smithi100 bash[18597]: cluster 2024-09-10T18:46:41.781123+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-09-10T18:46:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:42 smithi100 bash[18597]: cluster 2024-09-10T18:46:42.049965+0000 mgr.y (mgr.24512) 486 : cluster [DBG] pgmap v754: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:42 smithi100 bash[31607]: audit 2024-09-10T18:46:41.780994+0000 mon.a (mon.0) 2735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:46:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:42 smithi100 bash[31607]: cluster 2024-09-10T18:46:41.781123+0000 mon.a (mon.0) 2736 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-09-10T18:46:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:42 smithi100 bash[31607]: cluster 2024-09-10T18:46:42.049965+0000 mgr.y (mgr.24512) 486 : cluster [DBG] pgmap v754: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[31607]: audit 2024-09-10T18:46:42.564943+0000 mgr.y (mgr.24512) 487 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[31607]: cluster 2024-09-10T18:46:42.780040+0000 mon.a (mon.0) 2737 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[31607]: audit 2024-09-10T18:46:42.814857+0000 mon.b (mon.1) 348 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[31607]: audit 2024-09-10T18:46:42.816232+0000 mon.b (mon.1) 349 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-87"}]: dispatch 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[31607]: audit 2024-09-10T18:46:42.825528+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[31607]: audit 2024-09-10T18:46:42.826714+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-87"}]: dispatch 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[18597]: audit 2024-09-10T18:46:42.564943+0000 mgr.y (mgr.24512) 487 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:44.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[18597]: cluster 2024-09-10T18:46:42.780040+0000 mon.a (mon.0) 2737 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-09-10T18:46:44.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[18597]: audit 2024-09-10T18:46:42.814857+0000 mon.b (mon.1) 348 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:44.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[18597]: audit 2024-09-10T18:46:42.816232+0000 mon.b (mon.1) 349 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-87"}]: dispatch 2024-09-10T18:46:44.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[18597]: audit 2024-09-10T18:46:42.825528+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:44.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:43 smithi100 bash[18597]: audit 2024-09-10T18:46:42.826714+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-87"}]: dispatch 2024-09-10T18:46:44.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:43 smithi151 bash[22565]: audit 2024-09-10T18:46:42.564943+0000 mgr.y (mgr.24512) 487 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:44.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:43 smithi151 bash[22565]: cluster 2024-09-10T18:46:42.780040+0000 mon.a (mon.0) 2737 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-09-10T18:46:44.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:43 smithi151 bash[22565]: audit 2024-09-10T18:46:42.814857+0000 mon.b (mon.1) 348 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:44.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:43 smithi151 bash[22565]: audit 2024-09-10T18:46:42.816232+0000 mon.b (mon.1) 349 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-87"}]: dispatch 2024-09-10T18:46:44.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:43 smithi151 bash[22565]: audit 2024-09-10T18:46:42.825528+0000 mon.a (mon.0) 2738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:44.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:43 smithi151 bash[22565]: audit 2024-09-10T18:46:42.826714+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-87"}]: dispatch 2024-09-10T18:46:44.811 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:46:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:44.812 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:44 smithi151 bash[22565]: cluster 2024-09-10T18:46:43.801144+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-09-10T18:46:44.812 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:44 smithi151 bash[22565]: cluster 2024-09-10T18:46:44.050812+0000 mgr.y (mgr.24512) 488 : cluster [DBG] pgmap v757: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:44 smithi100 bash[18597]: cluster 2024-09-10T18:46:43.801144+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-09-10T18:46:45.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:44 smithi100 bash[18597]: cluster 2024-09-10T18:46:44.050812+0000 mgr.y (mgr.24512) 488 : cluster [DBG] pgmap v757: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:44 smithi100 bash[31607]: cluster 2024-09-10T18:46:43.801144+0000 mon.a (mon.0) 2740 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-09-10T18:46:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:44 smithi100 bash[31607]: cluster 2024-09-10T18:46:44.050812+0000 mgr.y (mgr.24512) 488 : cluster [DBG] pgmap v757: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:46:46.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: cluster 2024-09-10T18:46:44.796337+0000 mon.a (mon.0) 2741 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:46.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: cluster 2024-09-10T18:46:44.820440+0000 mon.a (mon.0) 2742 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-09-10T18:46:46.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: audit 2024-09-10T18:46:44.821678+0000 mon.b (mon.1) 350 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:46.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: audit 2024-09-10T18:46:44.833362+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:46.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: audit 2024-09-10T18:46:45.315149+0000 mon.a (mon.0) 2744 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: audit 2024-09-10T18:46:45.329294+0000 mon.a (mon.0) 2745 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: audit 2024-09-10T18:46:45.787632+0000 mon.a (mon.0) 2746 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.174 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:45 smithi151 bash[22565]: audit 2024-09-10T18:46:45.801151+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: cluster 2024-09-10T18:46:44.796337+0000 mon.a (mon.0) 2741 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: cluster 2024-09-10T18:46:44.820440+0000 mon.a (mon.0) 2742 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: audit 2024-09-10T18:46:44.821678+0000 mon.b (mon.1) 350 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: audit 2024-09-10T18:46:44.833362+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: audit 2024-09-10T18:46:45.315149+0000 mon.a (mon.0) 2744 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: audit 2024-09-10T18:46:45.329294+0000 mon.a (mon.0) 2745 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: audit 2024-09-10T18:46:45.787632+0000 mon.a (mon.0) 2746 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[18597]: audit 2024-09-10T18:46:45.801151+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: cluster 2024-09-10T18:46:44.796337+0000 mon.a (mon.0) 2741 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: cluster 2024-09-10T18:46:44.820440+0000 mon.a (mon.0) 2742 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: audit 2024-09-10T18:46:44.821678+0000 mon.b (mon.1) 350 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: audit 2024-09-10T18:46:44.833362+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: audit 2024-09-10T18:46:45.315149+0000 mon.a (mon.0) 2744 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: audit 2024-09-10T18:46:45.329294+0000 mon.a (mon.0) 2745 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: audit 2024-09-10T18:46:45.787632+0000 mon.a (mon.0) 2746 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:46.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:45 smithi100 bash[31607]: audit 2024-09-10T18:46:45.801151+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: audit 2024-09-10T18:46:45.818114+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: cluster 2024-09-10T18:46:45.818226+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: audit 2024-09-10T18:46:45.827643+0000 mon.b (mon.1) 351 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: audit 2024-09-10T18:46:45.831302+0000 mon.b (mon.1) 352 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: audit 2024-09-10T18:46:45.841941+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: cluster 2024-09-10T18:46:46.051692+0000 mgr.y (mgr.24512) 489 : cluster [DBG] pgmap v760: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: audit 2024-09-10T18:46:46.177021+0000 mon.c (mon.2) 465 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: audit 2024-09-10T18:46:46.178971+0000 mon.c (mon.2) 466 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:47.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:46 smithi151 bash[22565]: audit 2024-09-10T18:46:46.189843+0000 mon.a (mon.0) 2751 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: audit 2024-09-10T18:46:45.818114+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: cluster 2024-09-10T18:46:45.818226+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-09-10T18:46:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: audit 2024-09-10T18:46:45.827643+0000 mon.b (mon.1) 351 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: audit 2024-09-10T18:46:45.831302+0000 mon.b (mon.1) 352 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: audit 2024-09-10T18:46:45.841941+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:47.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: cluster 2024-09-10T18:46:46.051692+0000 mgr.y (mgr.24512) 489 : cluster [DBG] pgmap v760: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: audit 2024-09-10T18:46:46.177021+0000 mon.c (mon.2) 465 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: audit 2024-09-10T18:46:46.178971+0000 mon.c (mon.2) 466 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[18597]: audit 2024-09-10T18:46:46.189843+0000 mon.a (mon.0) 2751 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: audit 2024-09-10T18:46:45.818114+0000 mon.a (mon.0) 2748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: cluster 2024-09-10T18:46:45.818226+0000 mon.a (mon.0) 2749 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: audit 2024-09-10T18:46:45.827643+0000 mon.b (mon.1) 351 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: audit 2024-09-10T18:46:45.831302+0000 mon.b (mon.1) 352 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: audit 2024-09-10T18:46:45.841941+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: cluster 2024-09-10T18:46:46.051692+0000 mgr.y (mgr.24512) 489 : cluster [DBG] pgmap v760: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: audit 2024-09-10T18:46:46.177021+0000 mon.c (mon.2) 465 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:47.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: audit 2024-09-10T18:46:46.178971+0000 mon.c (mon.2) 466 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:47.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:46 smithi100 bash[31607]: audit 2024-09-10T18:46:46.189843+0000 mon.a (mon.0) 2751 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:46:48.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:47 smithi151 bash[22565]: audit 2024-09-10T18:46:46.842820+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:48.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:47 smithi151 bash[22565]: cluster 2024-09-10T18:46:46.842914+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-09-10T18:46:48.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:47 smithi100 bash[31607]: audit 2024-09-10T18:46:46.842820+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:47 smithi100 bash[31607]: cluster 2024-09-10T18:46:46.842914+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-09-10T18:46:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:47 smithi100 bash[18597]: audit 2024-09-10T18:46:46.842820+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:48.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:47 smithi100 bash[18597]: cluster 2024-09-10T18:46:46.842914+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-09-10T18:46:49.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:48 smithi151 bash[22565]: cluster 2024-09-10T18:46:47.849809+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-09-10T18:46:49.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:48 smithi151 bash[22565]: cluster 2024-09-10T18:46:48.052523+0000 mgr.y (mgr.24512) 490 : cluster [DBG] pgmap v763: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:46:49.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:48 smithi100 bash[31607]: cluster 2024-09-10T18:46:47.849809+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-09-10T18:46:49.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:48 smithi100 bash[31607]: cluster 2024-09-10T18:46:48.052523+0000 mgr.y (mgr.24512) 490 : cluster [DBG] pgmap v763: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:46:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:48 smithi100 bash[18597]: cluster 2024-09-10T18:46:47.849809+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-09-10T18:46:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:48 smithi100 bash[18597]: cluster 2024-09-10T18:46:48.052523+0000 mgr.y (mgr.24512) 490 : cluster [DBG] pgmap v763: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:46:50.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:49 smithi151 bash[22565]: cluster 2024-09-10T18:46:48.868795+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-09-10T18:46:50.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:49 smithi151 bash[22565]: audit 2024-09-10T18:46:48.920407+0000 mon.b (mon.1) 353 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:50.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:49 smithi151 bash[22565]: audit 2024-09-10T18:46:48.922464+0000 mon.b (mon.1) 354 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-90"}]: dispatch 2024-09-10T18:46:50.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:49 smithi151 bash[22565]: audit 2024-09-10T18:46:48.931201+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:50.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:49 smithi151 bash[22565]: audit 2024-09-10T18:46:48.933227+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-90"}]: dispatch 2024-09-10T18:46:50.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[18597]: cluster 2024-09-10T18:46:48.868795+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[18597]: audit 2024-09-10T18:46:48.920407+0000 mon.b (mon.1) 353 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[18597]: audit 2024-09-10T18:46:48.922464+0000 mon.b (mon.1) 354 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-90"}]: dispatch 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[18597]: audit 2024-09-10T18:46:48.931201+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[18597]: audit 2024-09-10T18:46:48.933227+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-90"}]: dispatch 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[31607]: cluster 2024-09-10T18:46:48.868795+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[31607]: audit 2024-09-10T18:46:48.920407+0000 mon.b (mon.1) 353 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[31607]: audit 2024-09-10T18:46:48.922464+0000 mon.b (mon.1) 354 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-90"}]: dispatch 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[31607]: audit 2024-09-10T18:46:48.931201+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:49 smithi100 bash[31607]: audit 2024-09-10T18:46:48.933227+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-90"}]: dispatch 2024-09-10T18:46:51.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:50 smithi151 bash[22565]: cluster 2024-09-10T18:46:49.879773+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-09-10T18:46:51.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:50 smithi151 bash[22565]: cluster 2024-09-10T18:46:50.053088+0000 mgr.y (mgr.24512) 491 : cluster [DBG] pgmap v766: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:51.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:50 smithi100 bash[31607]: cluster 2024-09-10T18:46:49.879773+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-09-10T18:46:51.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:50 smithi100 bash[31607]: cluster 2024-09-10T18:46:50.053088+0000 mgr.y (mgr.24512) 491 : cluster [DBG] pgmap v766: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:51.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:50 smithi100 bash[18597]: cluster 2024-09-10T18:46:49.879773+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-09-10T18:46:51.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:50 smithi100 bash[18597]: cluster 2024-09-10T18:46:50.053088+0000 mgr.y (mgr.24512) 491 : cluster [DBG] pgmap v766: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:46:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:51] "GET /metrics HTTP/1.1" 200 231064 "" "Prometheus/2.43.0" 2024-09-10T18:46:52.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:51 smithi151 bash[22565]: audit 2024-09-10T18:46:50.889364+0000 mon.b (mon.1) 355 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:52.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:51 smithi151 bash[22565]: cluster 2024-09-10T18:46:50.890588+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-09-10T18:46:52.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:51 smithi151 bash[22565]: audit 2024-09-10T18:46:50.900599+0000 mon.a (mon.0) 2760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:52.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:51 smithi100 bash[31607]: audit 2024-09-10T18:46:50.889364+0000 mon.b (mon.1) 355 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:51 smithi100 bash[31607]: cluster 2024-09-10T18:46:50.890588+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-09-10T18:46:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:51 smithi100 bash[31607]: audit 2024-09-10T18:46:50.900599+0000 mon.a (mon.0) 2760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:51 smithi100 bash[18597]: audit 2024-09-10T18:46:50.889364+0000 mon.b (mon.1) 355 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:51 smithi100 bash[18597]: cluster 2024-09-10T18:46:50.890588+0000 mon.a (mon.0) 2759 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-09-10T18:46:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:51 smithi100 bash[18597]: audit 2024-09-10T18:46:50.900599+0000 mon.a (mon.0) 2760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:52.898 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:46:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:46:53.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:52 smithi151 bash[22565]: audit 2024-09-10T18:46:51.896442+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:53.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:52 smithi151 bash[22565]: audit 2024-09-10T18:46:51.903297+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:53.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:52 smithi151 bash[22565]: cluster 2024-09-10T18:46:51.903661+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-09-10T18:46:53.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:52 smithi151 bash[22565]: cluster 2024-09-10T18:46:52.053914+0000 mgr.y (mgr.24512) 492 : cluster [DBG] pgmap v769: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[31607]: audit 2024-09-10T18:46:51.896442+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[31607]: audit 2024-09-10T18:46:51.903297+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[31607]: cluster 2024-09-10T18:46:51.903661+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[31607]: cluster 2024-09-10T18:46:52.053914+0000 mgr.y (mgr.24512) 492 : cluster [DBG] pgmap v769: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[18597]: audit 2024-09-10T18:46:51.896442+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[18597]: audit 2024-09-10T18:46:51.903297+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[18597]: cluster 2024-09-10T18:46:51.903661+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-09-10T18:46:53.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:52 smithi100 bash[18597]: cluster 2024-09-10T18:46:52.053914+0000 mgr.y (mgr.24512) 492 : cluster [DBG] pgmap v769: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:46:54.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:53 smithi151 bash[22565]: audit 2024-09-10T18:46:52.576185+0000 mgr.y (mgr.24512) 493 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:54.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:53 smithi151 bash[22565]: cluster 2024-09-10T18:46:52.912358+0000 mon.a (mon.0) 2763 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-09-10T18:46:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:53 smithi100 bash[18597]: audit 2024-09-10T18:46:52.576185+0000 mgr.y (mgr.24512) 493 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:53 smithi100 bash[18597]: cluster 2024-09-10T18:46:52.912358+0000 mon.a (mon.0) 2763 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-09-10T18:46:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:53 smithi100 bash[31607]: audit 2024-09-10T18:46:52.576185+0000 mgr.y (mgr.24512) 493 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:53 smithi100 bash[31607]: cluster 2024-09-10T18:46:52.912358+0000 mon.a (mon.0) 2763 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-09-10T18:46:54.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:46:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[31607]: cluster 2024-09-10T18:46:53.916655+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-09-10T18:46:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[31607]: audit 2024-09-10T18:46:53.958643+0000 mon.b (mon.1) 357 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[31607]: audit 2024-09-10T18:46:53.960562+0000 mon.b (mon.1) 358 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-92"}]: dispatch 2024-09-10T18:46:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[31607]: audit 2024-09-10T18:46:53.969267+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[31607]: audit 2024-09-10T18:46:53.970962+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-92"}]: dispatch 2024-09-10T18:46:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[31607]: cluster 2024-09-10T18:46:54.054756+0000 mgr.y (mgr.24512) 494 : cluster [DBG] pgmap v772: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:46:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[31607]: audit 2024-09-10T18:46:54.335193+0000 mon.c (mon.2) 467 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[18597]: cluster 2024-09-10T18:46:53.916655+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-09-10T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[18597]: audit 2024-09-10T18:46:53.958643+0000 mon.b (mon.1) 357 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[18597]: audit 2024-09-10T18:46:53.960562+0000 mon.b (mon.1) 358 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-92"}]: dispatch 2024-09-10T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[18597]: audit 2024-09-10T18:46:53.969267+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[18597]: audit 2024-09-10T18:46:53.970962+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-92"}]: dispatch 2024-09-10T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[18597]: cluster 2024-09-10T18:46:54.054756+0000 mgr.y (mgr.24512) 494 : cluster [DBG] pgmap v772: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:54 smithi100 bash[18597]: audit 2024-09-10T18:46:54.335193+0000 mon.c (mon.2) 467 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:55.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[22565]: cluster 2024-09-10T18:46:53.916655+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-09-10T18:46:55.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[22565]: audit 2024-09-10T18:46:53.958643+0000 mon.b (mon.1) 357 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:55.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[22565]: audit 2024-09-10T18:46:53.960562+0000 mon.b (mon.1) 358 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-92"}]: dispatch 2024-09-10T18:46:55.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[22565]: audit 2024-09-10T18:46:53.969267+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:46:55.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[22565]: audit 2024-09-10T18:46:53.970962+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-92"}]: dispatch 2024-09-10T18:46:55.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[22565]: cluster 2024-09-10T18:46:54.054756+0000 mgr.y (mgr.24512) 494 : cluster [DBG] pgmap v772: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:46:55.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:54 smithi151 bash[22565]: audit 2024-09-10T18:46:54.335193+0000 mon.c (mon.2) 467 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:56.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:55 smithi100 bash[18597]: cluster 2024-09-10T18:46:54.936611+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-09-10T18:46:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:55 smithi100 bash[31607]: cluster 2024-09-10T18:46:54.936611+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-09-10T18:46:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:55 smithi151 bash[22565]: cluster 2024-09-10T18:46:54.936611+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-09-10T18:46:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[18597]: audit 2024-09-10T18:46:55.944177+0000 mon.b (mon.1) 359 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[18597]: cluster 2024-09-10T18:46:55.950851+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-09-10T18:46:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[18597]: audit 2024-09-10T18:46:55.956034+0000 mon.a (mon.0) 2769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[18597]: cluster 2024-09-10T18:46:56.055777+0000 mgr.y (mgr.24512) 495 : cluster [DBG] pgmap v775: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:46:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[31607]: audit 2024-09-10T18:46:55.944177+0000 mon.b (mon.1) 359 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[31607]: cluster 2024-09-10T18:46:55.950851+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-09-10T18:46:57.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[31607]: audit 2024-09-10T18:46:55.956034+0000 mon.a (mon.0) 2769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:57.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:56 smithi100 bash[31607]: cluster 2024-09-10T18:46:56.055777+0000 mgr.y (mgr.24512) 495 : cluster [DBG] pgmap v775: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:46:57.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:56 smithi151 bash[22565]: audit 2024-09-10T18:46:55.944177+0000 mon.b (mon.1) 359 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:57.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:56 smithi151 bash[22565]: cluster 2024-09-10T18:46:55.950851+0000 mon.a (mon.0) 2768 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-09-10T18:46:57.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:56 smithi151 bash[22565]: audit 2024-09-10T18:46:55.956034+0000 mon.a (mon.0) 2769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:57.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:56 smithi151 bash[22565]: cluster 2024-09-10T18:46:56.055777+0000 mgr.y (mgr.24512) 495 : cluster [DBG] pgmap v775: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:46:58.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[18597]: cluster 2024-09-10T18:46:56.945844+0000 mon.a (mon.0) 2770 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[18597]: audit 2024-09-10T18:46:56.964777+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[18597]: cluster 2024-09-10T18:46:56.964990+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-09-10T18:46:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[18597]: audit 2024-09-10T18:46:56.965603+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[31607]: cluster 2024-09-10T18:46:56.945844+0000 mon.a (mon.0) 2770 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[31607]: audit 2024-09-10T18:46:56.964777+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[31607]: cluster 2024-09-10T18:46:56.964990+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-09-10T18:46:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:57 smithi100 bash[31607]: audit 2024-09-10T18:46:56.965603+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:57 smithi151 bash[22565]: cluster 2024-09-10T18:46:56.945844+0000 mon.a (mon.0) 2770 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:57 smithi151 bash[22565]: audit 2024-09-10T18:46:56.964777+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:57 smithi151 bash[22565]: cluster 2024-09-10T18:46:56.964990+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-09-10T18:46:58.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:57 smithi151 bash[22565]: audit 2024-09-10T18:46:56.965603+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:58 smithi100 bash[18597]: cluster 2024-09-10T18:46:57.971251+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-09-10T18:46:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:58 smithi100 bash[18597]: cluster 2024-09-10T18:46:58.056659+0000 mgr.y (mgr.24512) 496 : cluster [DBG] pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:46:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:58 smithi100 bash[31607]: cluster 2024-09-10T18:46:57.971251+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-09-10T18:46:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:58 smithi100 bash[31607]: cluster 2024-09-10T18:46:58.056659+0000 mgr.y (mgr.24512) 496 : cluster [DBG] pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:46:59.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:58 smithi151 bash[22565]: cluster 2024-09-10T18:46:57.971251+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-09-10T18:46:59.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:58 smithi151 bash[22565]: cluster 2024-09-10T18:46:58.056659+0000 mgr.y (mgr.24512) 496 : cluster [DBG] pgmap v778: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:47:00.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:46:59 smithi100 bash[18597]: cluster 2024-09-10T18:46:58.979036+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-09-10T18:47:00.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:46:59 smithi100 bash[31607]: cluster 2024-09-10T18:46:58.979036+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-09-10T18:47:00.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:46:59 smithi151 bash[22565]: cluster 2024-09-10T18:46:58.979036+0000 mon.a (mon.0) 2774 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-09-10T18:47:01.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:00 smithi100 bash[18597]: cluster 2024-09-10T18:47:00.057557+0000 mgr.y (mgr.24512) 497 : cluster [DBG] pgmap v780: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:01.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:00 smithi100 bash[31607]: cluster 2024-09-10T18:47:00.057557+0000 mgr.y (mgr.24512) 497 : cluster [DBG] pgmap v780: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:01.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:00 smithi151 bash[22565]: cluster 2024-09-10T18:47:00.057557+0000 mgr.y (mgr.24512) 497 : cluster [DBG] pgmap v780: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:47:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:01] "GET /metrics HTTP/1.1" 200 228682 "" "Prometheus/2.43.0" 2024-09-10T18:47:02.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:02 smithi151 bash[22565]: cluster 2024-09-10T18:47:02.059728+0000 mgr.y (mgr.24512) 498 : cluster [DBG] pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T18:47:02.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:02 smithi100 bash[18597]: cluster 2024-09-10T18:47:02.059728+0000 mgr.y (mgr.24512) 498 : cluster [DBG] pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T18:47:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:02 smithi100 bash[31607]: cluster 2024-09-10T18:47:02.059728+0000 mgr.y (mgr.24512) 498 : cluster [DBG] pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T18:47:02.922 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:47:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:47:03.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:03 smithi151 bash[22565]: audit 2024-09-10T18:47:02.587501+0000 mgr.y (mgr.24512) 499 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:03 smithi100 bash[18597]: audit 2024-09-10T18:47:02.587501+0000 mgr.y (mgr.24512) 499 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:03 smithi100 bash[31607]: audit 2024-09-10T18:47:02.587501+0000 mgr.y (mgr.24512) 499 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:04.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:04 smithi151 bash[22565]: cluster 2024-09-10T18:47:04.060569+0000 mgr.y (mgr.24512) 500 : cluster [DBG] pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:47:04.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:04 smithi100 bash[18597]: cluster 2024-09-10T18:47:04.060569+0000 mgr.y (mgr.24512) 500 : cluster [DBG] pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:47:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:04 smithi100 bash[31607]: cluster 2024-09-10T18:47:04.060569+0000 mgr.y (mgr.24512) 500 : cluster [DBG] pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:47:04.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:47:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:06.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:06 smithi151 bash[22565]: cluster 2024-09-10T18:47:06.058843+0000 mon.a (mon.0) 2775 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:06.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:06 smithi151 bash[22565]: cluster 2024-09-10T18:47:06.062717+0000 mgr.y (mgr.24512) 501 : cluster [DBG] pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1012 B/s wr, 4 op/s 2024-09-10T18:47:06.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:06 smithi100 bash[18597]: cluster 2024-09-10T18:47:06.058843+0000 mon.a (mon.0) 2775 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:06 smithi100 bash[18597]: cluster 2024-09-10T18:47:06.062717+0000 mgr.y (mgr.24512) 501 : cluster [DBG] pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1012 B/s wr, 4 op/s 2024-09-10T18:47:06.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:06 smithi100 bash[31607]: cluster 2024-09-10T18:47:06.058843+0000 mon.a (mon.0) 2775 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:06.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:06 smithi100 bash[31607]: cluster 2024-09-10T18:47:06.062717+0000 mgr.y (mgr.24512) 501 : cluster [DBG] pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1012 B/s wr, 4 op/s 2024-09-10T18:47:08.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:08 smithi151 bash[22565]: cluster 2024-09-10T18:47:08.063554+0000 mgr.y (mgr.24512) 502 : cluster [DBG] pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:47:08.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:08 smithi100 bash[18597]: cluster 2024-09-10T18:47:08.063554+0000 mgr.y (mgr.24512) 502 : cluster [DBG] pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:47:08.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:08 smithi100 bash[31607]: cluster 2024-09-10T18:47:08.063554+0000 mgr.y (mgr.24512) 502 : cluster [DBG] pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:47:10.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:10 smithi151 bash[22565]: cluster 2024-09-10T18:47:09.156890+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-09-10T18:47:10.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:10 smithi151 bash[22565]: audit 2024-09-10T18:47:09.334620+0000 mon.c (mon.2) 468 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:10.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:10 smithi151 bash[22565]: cluster 2024-09-10T18:47:10.064362+0000 mgr.y (mgr.24512) 503 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:47:10.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:10 smithi100 bash[18597]: cluster 2024-09-10T18:47:09.156890+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-09-10T18:47:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:10 smithi100 bash[18597]: audit 2024-09-10T18:47:09.334620+0000 mon.c (mon.2) 468 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:10 smithi100 bash[18597]: cluster 2024-09-10T18:47:10.064362+0000 mgr.y (mgr.24512) 503 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:47:10.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:10 smithi100 bash[31607]: cluster 2024-09-10T18:47:09.156890+0000 mon.a (mon.0) 2776 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-09-10T18:47:10.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:10 smithi100 bash[31607]: audit 2024-09-10T18:47:09.334620+0000 mon.c (mon.2) 468 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:10.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:10 smithi100 bash[31607]: cluster 2024-09-10T18:47:10.064362+0000 mgr.y (mgr.24512) 503 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:47:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:47:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:11] "GET /metrics HTTP/1.1" 200 228635 "" "Prometheus/2.43.0" 2024-09-10T18:47:12.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:12 smithi151 bash[22565]: cluster 2024-09-10T18:47:12.066614+0000 mgr.y (mgr.24512) 504 : cluster [DBG] pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:12.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:12 smithi100 bash[18597]: cluster 2024-09-10T18:47:12.066614+0000 mgr.y (mgr.24512) 504 : cluster [DBG] pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:12.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:12 smithi100 bash[31607]: cluster 2024-09-10T18:47:12.066614+0000 mgr.y (mgr.24512) 504 : cluster [DBG] pgmap v787: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:12.922 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:47:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:47:13.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:13 smithi151 bash[22565]: audit 2024-09-10T18:47:12.598800+0000 mgr.y (mgr.24512) 505 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:13.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:13 smithi100 bash[18597]: audit 2024-09-10T18:47:12.598800+0000 mgr.y (mgr.24512) 505 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:13.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:13 smithi100 bash[31607]: audit 2024-09-10T18:47:12.598800+0000 mgr.y (mgr.24512) 505 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:14.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:14 smithi151 bash[22565]: cluster 2024-09-10T18:47:14.067457+0000 mgr.y (mgr.24512) 506 : cluster [DBG] pgmap v788: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:14.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:14 smithi100 bash[18597]: cluster 2024-09-10T18:47:14.067457+0000 mgr.y (mgr.24512) 506 : cluster [DBG] pgmap v788: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:14.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:14 smithi100 bash[31607]: cluster 2024-09-10T18:47:14.067457+0000 mgr.y (mgr.24512) 506 : cluster [DBG] pgmap v788: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:14.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:47:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:17.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:17 smithi151 bash[22565]: cluster 2024-09-10T18:47:16.068783+0000 mgr.y (mgr.24512) 507 : cluster [DBG] pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:17.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:17 smithi151 bash[22565]: cluster 2024-09-10T18:47:16.070460+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-09-10T18:47:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:17 smithi100 bash[18597]: cluster 2024-09-10T18:47:16.068783+0000 mgr.y (mgr.24512) 507 : cluster [DBG] pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:17.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:17 smithi100 bash[18597]: cluster 2024-09-10T18:47:16.070460+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-09-10T18:47:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:17 smithi100 bash[31607]: cluster 2024-09-10T18:47:16.068783+0000 mgr.y (mgr.24512) 507 : cluster [DBG] pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:17 smithi100 bash[31607]: cluster 2024-09-10T18:47:16.070460+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-09-10T18:47:19.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:19 smithi151 bash[22565]: cluster 2024-09-10T18:47:18.069675+0000 mgr.y (mgr.24512) 508 : cluster [DBG] pgmap v791: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:19.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:19 smithi100 bash[18597]: cluster 2024-09-10T18:47:18.069675+0000 mgr.y (mgr.24512) 508 : cluster [DBG] pgmap v791: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:19.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:19 smithi100 bash[31607]: cluster 2024-09-10T18:47:18.069675+0000 mgr.y (mgr.24512) 508 : cluster [DBG] pgmap v791: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:20.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:20 smithi151 bash[22565]: audit 2024-09-10T18:47:19.190374+0000 mon.b (mon.1) 361 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:20.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:20 smithi151 bash[22565]: audit 2024-09-10T18:47:19.192621+0000 mon.b (mon.1) 362 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-94"}]: dispatch 2024-09-10T18:47:20.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:20 smithi151 bash[22565]: audit 2024-09-10T18:47:19.200673+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:20.423 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:20 smithi151 bash[22565]: audit 2024-09-10T18:47:19.202800+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-94"}]: dispatch 2024-09-10T18:47:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[18597]: audit 2024-09-10T18:47:19.190374+0000 mon.b (mon.1) 361 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[18597]: audit 2024-09-10T18:47:19.192621+0000 mon.b (mon.1) 362 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-94"}]: dispatch 2024-09-10T18:47:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[18597]: audit 2024-09-10T18:47:19.200673+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[18597]: audit 2024-09-10T18:47:19.202800+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-94"}]: dispatch 2024-09-10T18:47:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[31607]: audit 2024-09-10T18:47:19.190374+0000 mon.b (mon.1) 361 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[31607]: audit 2024-09-10T18:47:19.192621+0000 mon.b (mon.1) 362 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-94"}]: dispatch 2024-09-10T18:47:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[31607]: audit 2024-09-10T18:47:19.200673+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:20.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:20 smithi100 bash[31607]: audit 2024-09-10T18:47:19.202800+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-94"}]: dispatch 2024-09-10T18:47:21.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:21 smithi151 bash[22565]: cluster 2024-09-10T18:47:20.070599+0000 mgr.y (mgr.24512) 509 : cluster [DBG] pgmap v792: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:21.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:21 smithi151 bash[22565]: cluster 2024-09-10T18:47:20.099500+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-09-10T18:47:21.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:47:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:21] "GET /metrics HTTP/1.1" 200 228635 "" "Prometheus/2.43.0" 2024-09-10T18:47:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:21 smithi100 bash[18597]: cluster 2024-09-10T18:47:20.070599+0000 mgr.y (mgr.24512) 509 : cluster [DBG] pgmap v792: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:21.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:21 smithi100 bash[18597]: cluster 2024-09-10T18:47:20.099500+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-09-10T18:47:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:21 smithi100 bash[31607]: cluster 2024-09-10T18:47:20.070599+0000 mgr.y (mgr.24512) 509 : cluster [DBG] pgmap v792: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:21 smithi100 bash[31607]: cluster 2024-09-10T18:47:20.099500+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-09-10T18:47:22.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:22 smithi151 bash[22565]: audit 2024-09-10T18:47:21.104485+0000 mon.b (mon.1) 363 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:22.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:22 smithi151 bash[22565]: cluster 2024-09-10T18:47:21.112082+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-09-10T18:47:22.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:22 smithi151 bash[22565]: audit 2024-09-10T18:47:21.118257+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:22.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:22 smithi151 bash[22565]: cluster 2024-09-10T18:47:22.071490+0000 mgr.y (mgr.24512) 510 : cluster [DBG] pgmap v795: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[18597]: audit 2024-09-10T18:47:21.104485+0000 mon.b (mon.1) 363 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:22.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[18597]: cluster 2024-09-10T18:47:21.112082+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-09-10T18:47:22.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[18597]: audit 2024-09-10T18:47:21.118257+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:22.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[18597]: cluster 2024-09-10T18:47:22.071490+0000 mgr.y (mgr.24512) 510 : cluster [DBG] pgmap v795: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[31607]: audit 2024-09-10T18:47:21.104485+0000 mon.b (mon.1) 363 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[31607]: cluster 2024-09-10T18:47:21.112082+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-09-10T18:47:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[31607]: audit 2024-09-10T18:47:21.118257+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:22 smithi100 bash[31607]: cluster 2024-09-10T18:47:22.071490+0000 mgr.y (mgr.24512) 510 : cluster [DBG] pgmap v795: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:22.922 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:47:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:47:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:23 smithi151 bash[22565]: cluster 2024-09-10T18:47:22.103638+0000 mon.a (mon.0) 2783 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:23 smithi151 bash[22565]: audit 2024-09-10T18:47:22.115159+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:23 smithi151 bash[22565]: audit 2024-09-10T18:47:22.119718+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:23 smithi151 bash[22565]: cluster 2024-09-10T18:47:22.121054+0000 mon.a (mon.0) 2785 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-09-10T18:47:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:23 smithi151 bash[22565]: audit 2024-09-10T18:47:22.610042+0000 mgr.y (mgr.24512) 511 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[18597]: cluster 2024-09-10T18:47:22.103638+0000 mon.a (mon.0) 2783 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[18597]: audit 2024-09-10T18:47:22.115159+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[18597]: audit 2024-09-10T18:47:22.119718+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[18597]: cluster 2024-09-10T18:47:22.121054+0000 mon.a (mon.0) 2785 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-09-10T18:47:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[18597]: audit 2024-09-10T18:47:22.610042+0000 mgr.y (mgr.24512) 511 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[31607]: cluster 2024-09-10T18:47:22.103638+0000 mon.a (mon.0) 2783 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[31607]: audit 2024-09-10T18:47:22.115159+0000 mon.b (mon.1) 364 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[31607]: audit 2024-09-10T18:47:22.119718+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[31607]: cluster 2024-09-10T18:47:22.121054+0000 mon.a (mon.0) 2785 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-09-10T18:47:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:23 smithi100 bash[31607]: audit 2024-09-10T18:47:22.610042+0000 mgr.y (mgr.24512) 511 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:24.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:24 smithi151 bash[22565]: cluster 2024-09-10T18:47:23.126950+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-09-10T18:47:24.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:24 smithi151 bash[22565]: cluster 2024-09-10T18:47:24.072338+0000 mgr.y (mgr.24512) 512 : cluster [DBG] pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:24.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:24 smithi100 bash[18597]: cluster 2024-09-10T18:47:23.126950+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-09-10T18:47:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:24 smithi100 bash[18597]: cluster 2024-09-10T18:47:24.072338+0000 mgr.y (mgr.24512) 512 : cluster [DBG] pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:24 smithi100 bash[31607]: cluster 2024-09-10T18:47:23.126950+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-09-10T18:47:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:24 smithi100 bash[31607]: cluster 2024-09-10T18:47:24.072338+0000 mgr.y (mgr.24512) 512 : cluster [DBG] pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:24.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:47:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:25.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:25 smithi151 bash[22565]: audit 2024-09-10T18:47:24.335857+0000 mon.c (mon.2) 469 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:25.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:25 smithi100 bash[18597]: audit 2024-09-10T18:47:24.335857+0000 mon.c (mon.2) 469 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:25.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:25 smithi100 bash[31607]: audit 2024-09-10T18:47:24.335857+0000 mon.c (mon.2) 469 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:26.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:26 smithi151 bash[22565]: cluster 2024-09-10T18:47:26.074550+0000 mgr.y (mgr.24512) 513 : cluster [DBG] pgmap v799: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-10T18:47:26.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:26 smithi100 bash[18597]: cluster 2024-09-10T18:47:26.074550+0000 mgr.y (mgr.24512) 513 : cluster [DBG] pgmap v799: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-10T18:47:26.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:26 smithi100 bash[31607]: cluster 2024-09-10T18:47:26.074550+0000 mgr.y (mgr.24512) 513 : cluster [DBG] pgmap v799: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-10T18:47:28.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:28 smithi151 bash[22565]: cluster 2024-09-10T18:47:28.075478+0000 mgr.y (mgr.24512) 514 : cluster [DBG] pgmap v800: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 735 B/s rd, 735 B/s wr, 2 op/s 2024-09-10T18:47:28.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:28 smithi100 bash[18597]: cluster 2024-09-10T18:47:28.075478+0000 mgr.y (mgr.24512) 514 : cluster [DBG] pgmap v800: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 735 B/s rd, 735 B/s wr, 2 op/s 2024-09-10T18:47:28.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:28 smithi100 bash[31607]: cluster 2024-09-10T18:47:28.075478+0000 mgr.y (mgr.24512) 514 : cluster [DBG] pgmap v800: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 735 B/s rd, 735 B/s wr, 2 op/s 2024-09-10T18:47:30.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:30 smithi151 bash[22565]: cluster 2024-09-10T18:47:30.076431+0000 mgr.y (mgr.24512) 515 : cluster [DBG] pgmap v801: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-10T18:47:30.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:30 smithi100 bash[18597]: cluster 2024-09-10T18:47:30.076431+0000 mgr.y (mgr.24512) 515 : cluster [DBG] pgmap v801: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-10T18:47:30.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:30 smithi100 bash[31607]: cluster 2024-09-10T18:47:30.076431+0000 mgr.y (mgr.24512) 515 : cluster [DBG] pgmap v801: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-09-10T18:47:31.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:31 smithi151 bash[22565]: cluster 2024-09-10T18:47:31.068437+0000 mon.a (mon.0) 2787 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:31.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:31 smithi100 bash[18597]: cluster 2024-09-10T18:47:31.068437+0000 mon.a (mon.0) 2787 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:31.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:47:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:31] "GET /metrics HTTP/1.1" 200 228604 "" "Prometheus/2.43.0" 2024-09-10T18:47:31.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:31 smithi100 bash[31607]: cluster 2024-09-10T18:47:31.068437+0000 mon.a (mon.0) 2787 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:32.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:32 smithi151 bash[22565]: cluster 2024-09-10T18:47:32.078747+0000 mgr.y (mgr.24512) 516 : cluster [DBG] pgmap v802: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 513 B/s wr, 2 op/s 2024-09-10T18:47:32.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:32 smithi100 bash[18597]: cluster 2024-09-10T18:47:32.078747+0000 mgr.y (mgr.24512) 516 : cluster [DBG] pgmap v802: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 513 B/s wr, 2 op/s 2024-09-10T18:47:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:32 smithi100 bash[31607]: cluster 2024-09-10T18:47:32.078747+0000 mgr.y (mgr.24512) 516 : cluster [DBG] pgmap v802: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 513 B/s wr, 2 op/s 2024-09-10T18:47:32.922 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:47:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:47:33.422 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:47:33 smithi151 bash[33542]: debug 2024-09-10T18:47:33.137+0000 7f3436bd3640 -1 snap_mapper.add_oid found existing snaps mapped on 65:bf3317c6:test-rados-api-smithi100-72764-97::foo:31, removing 2024-09-10T18:47:33.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:33 smithi151 bash[22565]: audit 2024-09-10T18:47:32.620287+0000 mgr.y (mgr.24512) 517 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:33.422 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:47:33 smithi151 bash[26435]: debug 2024-09-10T18:47:33.137+0000 7fc384dd5640 -1 snap_mapper.add_oid found existing snaps mapped on 65:bf3317c6:test-rados-api-smithi100-72764-97::foo:31, removing 2024-09-10T18:47:33.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:33 smithi100 bash[31607]: audit 2024-09-10T18:47:32.620287+0000 mgr.y (mgr.24512) 517 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:33 smithi100 bash[18597]: audit 2024-09-10T18:47:32.620287+0000 mgr.y (mgr.24512) 517 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:33.544 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:47:33 smithi100 bash[34936]: debug 2024-09-10T18:47:33.145+0000 7fc4cab5b640 -1 snap_mapper.add_oid found existing snaps mapped on 65:bf3317c6:test-rados-api-smithi100-72764-97::foo:31, removing 2024-09-10T18:47:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:34 smithi151 bash[22565]: audit 2024-09-10T18:47:33.169018+0000 mon.b (mon.1) 365 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:34 smithi151 bash[22565]: audit 2024-09-10T18:47:33.170711+0000 mon.b (mon.1) 366 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-96"}]: dispatch 2024-09-10T18:47:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:34 smithi151 bash[22565]: audit 2024-09-10T18:47:33.178955+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:34 smithi151 bash[22565]: audit 2024-09-10T18:47:33.180580+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-96"}]: dispatch 2024-09-10T18:47:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:34 smithi151 bash[22565]: cluster 2024-09-10T18:47:34.079706+0000 mgr.y (mgr.24512) 518 : cluster [DBG] pgmap v803: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 935 B/s rd, 467 B/s wr, 1 op/s 2024-09-10T18:47:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[31607]: audit 2024-09-10T18:47:33.169018+0000 mon.b (mon.1) 365 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[31607]: audit 2024-09-10T18:47:33.170711+0000 mon.b (mon.1) 366 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-96"}]: dispatch 2024-09-10T18:47:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[31607]: audit 2024-09-10T18:47:33.178955+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[31607]: audit 2024-09-10T18:47:33.180580+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-96"}]: dispatch 2024-09-10T18:47:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[31607]: cluster 2024-09-10T18:47:34.079706+0000 mgr.y (mgr.24512) 518 : cluster [DBG] pgmap v803: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 935 B/s rd, 467 B/s wr, 1 op/s 2024-09-10T18:47:34.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[18597]: audit 2024-09-10T18:47:33.169018+0000 mon.b (mon.1) 365 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:34.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[18597]: audit 2024-09-10T18:47:33.170711+0000 mon.b (mon.1) 366 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-96"}]: dispatch 2024-09-10T18:47:34.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[18597]: audit 2024-09-10T18:47:33.178955+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:34.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[18597]: audit 2024-09-10T18:47:33.180580+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-96"}]: dispatch 2024-09-10T18:47:34.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:34 smithi100 bash[18597]: cluster 2024-09-10T18:47:34.079706+0000 mgr.y (mgr.24512) 518 : cluster [DBG] pgmap v803: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 935 B/s rd, 467 B/s wr, 1 op/s 2024-09-10T18:47:34.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:47:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:35.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:35 smithi151 bash[22565]: cluster 2024-09-10T18:47:34.176384+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-09-10T18:47:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:35 smithi100 bash[18597]: cluster 2024-09-10T18:47:34.176384+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-09-10T18:47:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:35 smithi100 bash[31607]: cluster 2024-09-10T18:47:34.176384+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-09-10T18:47:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[18597]: audit 2024-09-10T18:47:35.188788+0000 mon.b (mon.1) 367 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[18597]: cluster 2024-09-10T18:47:35.191745+0000 mon.a (mon.0) 2791 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-09-10T18:47:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[18597]: audit 2024-09-10T18:47:35.198779+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[18597]: cluster 2024-09-10T18:47:36.081938+0000 mgr.y (mgr.24512) 519 : cluster [DBG] pgmap v806: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-09-10T18:47:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[31607]: audit 2024-09-10T18:47:35.188788+0000 mon.b (mon.1) 367 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[31607]: cluster 2024-09-10T18:47:35.191745+0000 mon.a (mon.0) 2791 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-09-10T18:47:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[31607]: audit 2024-09-10T18:47:35.198779+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:36 smithi100 bash[31607]: cluster 2024-09-10T18:47:36.081938+0000 mgr.y (mgr.24512) 519 : cluster [DBG] pgmap v806: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-09-10T18:47:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:36 smithi151 bash[22565]: audit 2024-09-10T18:47:35.188788+0000 mon.b (mon.1) 367 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:36 smithi151 bash[22565]: cluster 2024-09-10T18:47:35.191745+0000 mon.a (mon.0) 2791 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-09-10T18:47:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:36 smithi151 bash[22565]: audit 2024-09-10T18:47:35.198779+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:36 smithi151 bash[22565]: cluster 2024-09-10T18:47:36.081938+0000 mgr.y (mgr.24512) 519 : cluster [DBG] pgmap v806: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-09-10T18:47:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[18597]: cluster 2024-09-10T18:47:36.183875+0000 mon.a (mon.0) 2793 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[18597]: audit 2024-09-10T18:47:36.201985+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[18597]: cluster 2024-09-10T18:47:36.202261+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-09-10T18:47:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[18597]: audit 2024-09-10T18:47:36.205389+0000 mon.b (mon.1) 368 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[18597]: audit 2024-09-10T18:47:36.215703+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:37.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[31607]: cluster 2024-09-10T18:47:36.183875+0000 mon.a (mon.0) 2793 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:37.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[31607]: audit 2024-09-10T18:47:36.201985+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:37.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[31607]: cluster 2024-09-10T18:47:36.202261+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-09-10T18:47:37.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[31607]: audit 2024-09-10T18:47:36.205389+0000 mon.b (mon.1) 368 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:37.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:37 smithi100 bash[31607]: audit 2024-09-10T18:47:36.215703+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:37 smithi151 bash[22565]: cluster 2024-09-10T18:47:36.183875+0000 mon.a (mon.0) 2793 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:37 smithi151 bash[22565]: audit 2024-09-10T18:47:36.201985+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:37 smithi151 bash[22565]: cluster 2024-09-10T18:47:36.202261+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-09-10T18:47:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:37 smithi151 bash[22565]: audit 2024-09-10T18:47:36.205389+0000 mon.b (mon.1) 368 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:37 smithi151 bash[22565]: audit 2024-09-10T18:47:36.215703+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[18597]: audit 2024-09-10T18:47:37.195813+0000 mon.b (mon.1) 369 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]: dispatch 2024-09-10T18:47:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[18597]: audit 2024-09-10T18:47:37.203627+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:47:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[18597]: cluster 2024-09-10T18:47:37.203828+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-09-10T18:47:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[18597]: audit 2024-09-10T18:47:37.206500+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]: dispatch 2024-09-10T18:47:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[18597]: cluster 2024-09-10T18:47:38.082781+0000 mgr.y (mgr.24512) 520 : cluster [DBG] pgmap v809: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:47:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[18597]: cluster 2024-09-10T18:47:38.196452+0000 mon.a (mon.0) 2800 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:47:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[31607]: audit 2024-09-10T18:47:37.195813+0000 mon.b (mon.1) 369 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]: dispatch 2024-09-10T18:47:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[31607]: audit 2024-09-10T18:47:37.203627+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:47:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[31607]: cluster 2024-09-10T18:47:37.203828+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-09-10T18:47:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[31607]: audit 2024-09-10T18:47:37.206500+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]: dispatch 2024-09-10T18:47:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[31607]: cluster 2024-09-10T18:47:38.082781+0000 mgr.y (mgr.24512) 520 : cluster [DBG] pgmap v809: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:47:38.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:38 smithi100 bash[31607]: cluster 2024-09-10T18:47:38.196452+0000 mon.a (mon.0) 2800 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:47:38.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:38 smithi151 bash[22565]: audit 2024-09-10T18:47:37.195813+0000 mon.b (mon.1) 369 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]: dispatch 2024-09-10T18:47:38.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:38 smithi151 bash[22565]: audit 2024-09-10T18:47:37.203627+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:47:38.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:38 smithi151 bash[22565]: cluster 2024-09-10T18:47:37.203828+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-09-10T18:47:38.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:38 smithi151 bash[22565]: audit 2024-09-10T18:47:37.206500+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]: dispatch 2024-09-10T18:47:38.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:38 smithi151 bash[22565]: cluster 2024-09-10T18:47:38.082781+0000 mgr.y (mgr.24512) 520 : cluster [DBG] pgmap v809: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:47:38.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:38 smithi151 bash[22565]: cluster 2024-09-10T18:47:38.196452+0000 mon.a (mon.0) 2800 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:47:39.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[18597]: audit 2024-09-10T18:47:38.198023+0000 mon.b (mon.1) 370 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:39.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[18597]: audit 2024-09-10T18:47:38.205604+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]': finished 2024-09-10T18:47:39.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[18597]: cluster 2024-09-10T18:47:38.205894+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-09-10T18:47:39.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[18597]: audit 2024-09-10T18:47:38.208053+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[31607]: audit 2024-09-10T18:47:38.198023+0000 mon.b (mon.1) 370 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[31607]: audit 2024-09-10T18:47:38.205604+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]': finished 2024-09-10T18:47:39.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[31607]: cluster 2024-09-10T18:47:38.205894+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-09-10T18:47:39.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:39 smithi100 bash[31607]: audit 2024-09-10T18:47:38.208053+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:39.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:39 smithi151 bash[22565]: audit 2024-09-10T18:47:38.198023+0000 mon.b (mon.1) 370 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:39.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:39 smithi151 bash[22565]: audit 2024-09-10T18:47:38.205604+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-98", "mode": "writeback"}]': finished 2024-09-10T18:47:39.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:39 smithi151 bash[22565]: cluster 2024-09-10T18:47:38.205894+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-09-10T18:47:39.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:39 smithi151 bash[22565]: audit 2024-09-10T18:47:38.208053+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: audit 2024-09-10T18:47:39.200922+0000 mon.b (mon.1) 371 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:47:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: audit 2024-09-10T18:47:39.208548+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]': finished 2024-09-10T18:47:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: cluster 2024-09-10T18:47:39.208737+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-09-10T18:47:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: audit 2024-09-10T18:47:39.211987+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: audit 2024-09-10T18:47:39.343680+0000 mon.a (mon.0) 2807 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: audit 2024-09-10T18:47:39.346249+0000 mon.c (mon.2) 470 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: cluster 2024-09-10T18:47:40.083694+0000 mgr.y (mgr.24512) 521 : cluster [DBG] pgmap v812: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: cluster 2024-09-10T18:47:40.201979+0000 mon.a (mon.0) 2808 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: audit 2024-09-10T18:47:40.212170+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[18597]: cluster 2024-09-10T18:47:40.212402+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: audit 2024-09-10T18:47:39.200922+0000 mon.b (mon.1) 371 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: audit 2024-09-10T18:47:39.208548+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]': finished 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: cluster 2024-09-10T18:47:39.208737+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-09-10T18:47:40.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: audit 2024-09-10T18:47:39.211987+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:47:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: audit 2024-09-10T18:47:39.343680+0000 mon.a (mon.0) 2807 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:47:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: audit 2024-09-10T18:47:39.346249+0000 mon.c (mon.2) 470 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: cluster 2024-09-10T18:47:40.083694+0000 mgr.y (mgr.24512) 521 : cluster [DBG] pgmap v812: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: cluster 2024-09-10T18:47:40.201979+0000 mon.a (mon.0) 2808 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:47:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: audit 2024-09-10T18:47:40.212170+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:47:40.546 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:40 smithi100 bash[31607]: cluster 2024-09-10T18:47:40.212402+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-09-10T18:47:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: audit 2024-09-10T18:47:39.200922+0000 mon.b (mon.1) 371 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:47:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: audit 2024-09-10T18:47:39.208548+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-98"}]': finished 2024-09-10T18:47:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: cluster 2024-09-10T18:47:39.208737+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-09-10T18:47:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: audit 2024-09-10T18:47:39.211987+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:47:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: audit 2024-09-10T18:47:39.343680+0000 mon.a (mon.0) 2807 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:47:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: audit 2024-09-10T18:47:39.346249+0000 mon.c (mon.2) 470 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: cluster 2024-09-10T18:47:40.083694+0000 mgr.y (mgr.24512) 521 : cluster [DBG] pgmap v812: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:40.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: cluster 2024-09-10T18:47:40.201979+0000 mon.a (mon.0) 2808 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:47:40.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: audit 2024-09-10T18:47:40.212170+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:47:40.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:40 smithi151 bash[22565]: cluster 2024-09-10T18:47:40.212402+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-09-10T18:47:41.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:47:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:41] "GET /metrics HTTP/1.1" 200 228598 "" "Prometheus/2.43.0" 2024-09-10T18:47:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:41 smithi100 bash[18597]: audit 2024-09-10T18:47:40.204581+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:47:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:41 smithi100 bash[18597]: audit 2024-09-10T18:47:40.214871+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:47:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:41 smithi100 bash[18597]: audit 2024-09-10T18:47:41.215530+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T18:47:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:41 smithi100 bash[31607]: audit 2024-09-10T18:47:40.204581+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:47:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:41 smithi100 bash[31607]: audit 2024-09-10T18:47:40.214871+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:47:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:41 smithi100 bash[31607]: audit 2024-09-10T18:47:41.215530+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T18:47:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:41 smithi151 bash[22565]: audit 2024-09-10T18:47:40.204581+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:47:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:41 smithi151 bash[22565]: audit 2024-09-10T18:47:40.214871+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:47:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:41 smithi151 bash[22565]: audit 2024-09-10T18:47:41.215530+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T18:47:42.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: audit 2024-09-10T18:47:41.209698+0000 mon.b (mon.1) 373 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:42.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: cluster 2024-09-10T18:47:41.215695+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-09-10T18:47:42.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: audit 2024-09-10T18:47:41.221543+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:42.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: cluster 2024-09-10T18:47:42.084566+0000 mgr.y (mgr.24512) 522 : cluster [DBG] pgmap v815: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:42.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: audit 2024-09-10T18:47:42.210131+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:47:42.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: audit 2024-09-10T18:47:42.218103+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:47:42.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: cluster 2024-09-10T18:47:42.218222+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-09-10T18:47:42.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[18597]: audit 2024-09-10T18:47:42.220145+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:47:42.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: audit 2024-09-10T18:47:41.209698+0000 mon.b (mon.1) 373 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:42.548 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: cluster 2024-09-10T18:47:41.215695+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-09-10T18:47:42.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: audit 2024-09-10T18:47:41.221543+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:42.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: cluster 2024-09-10T18:47:42.084566+0000 mgr.y (mgr.24512) 522 : cluster [DBG] pgmap v815: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:42.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: audit 2024-09-10T18:47:42.210131+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:47:42.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: audit 2024-09-10T18:47:42.218103+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:47:42.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: cluster 2024-09-10T18:47:42.218222+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-09-10T18:47:42.549 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:42 smithi100 bash[31607]: audit 2024-09-10T18:47:42.220145+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:47:42.629 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: audit 2024-09-10T18:47:41.209698+0000 mon.b (mon.1) 373 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:42.629 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: cluster 2024-09-10T18:47:41.215695+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-09-10T18:47:42.629 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: audit 2024-09-10T18:47:41.221543+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:42.629 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: cluster 2024-09-10T18:47:42.084566+0000 mgr.y (mgr.24512) 522 : cluster [DBG] pgmap v815: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:42.629 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: audit 2024-09-10T18:47:42.210131+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:47:42.630 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: audit 2024-09-10T18:47:42.218103+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:47:42.630 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: cluster 2024-09-10T18:47:42.218222+0000 mon.a (mon.0) 2816 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-09-10T18:47:42.630 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[22565]: audit 2024-09-10T18:47:42.220145+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:47:42.922 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:47:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:47:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:43 smithi100 bash[18597]: audit 2024-09-10T18:47:42.628345+0000 mgr.y (mgr.24512) 523 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:43 smithi100 bash[18597]: audit 2024-09-10T18:47:43.223875+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T18:47:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:43 smithi100 bash[18597]: cluster 2024-09-10T18:47:43.224237+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-09-10T18:47:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:43 smithi100 bash[31607]: audit 2024-09-10T18:47:42.628345+0000 mgr.y (mgr.24512) 523 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:43 smithi100 bash[31607]: audit 2024-09-10T18:47:43.223875+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T18:47:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:43 smithi100 bash[31607]: cluster 2024-09-10T18:47:43.224237+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-09-10T18:47:43.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:43 smithi151 bash[22565]: audit 2024-09-10T18:47:42.628345+0000 mgr.y (mgr.24512) 523 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:43.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:43 smithi151 bash[22565]: audit 2024-09-10T18:47:43.223875+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T18:47:43.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:43 smithi151 bash[22565]: cluster 2024-09-10T18:47:43.224237+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-09-10T18:47:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:44 smithi100 bash[18597]: audit 2024-09-10T18:47:43.256502+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:44 smithi100 bash[18597]: audit 2024-09-10T18:47:43.266375+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:44 smithi100 bash[18597]: cluster 2024-09-10T18:47:44.085453+0000 mgr.y (mgr.24512) 524 : cluster [DBG] pgmap v818: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:44 smithi100 bash[31607]: audit 2024-09-10T18:47:43.256502+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:44 smithi100 bash[31607]: audit 2024-09-10T18:47:43.266375+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:44 smithi100 bash[31607]: cluster 2024-09-10T18:47:44.085453+0000 mgr.y (mgr.24512) 524 : cluster [DBG] pgmap v818: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:44.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:44 smithi151 bash[22565]: audit 2024-09-10T18:47:43.256502+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:44.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:44 smithi151 bash[22565]: audit 2024-09-10T18:47:43.266375+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:44.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:44 smithi151 bash[22565]: cluster 2024-09-10T18:47:44.085453+0000 mgr.y (mgr.24512) 524 : cluster [DBG] pgmap v818: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:44.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:47:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[18597]: audit 2024-09-10T18:47:44.248964+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[18597]: audit 2024-09-10T18:47:44.255122+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:47:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[18597]: cluster 2024-09-10T18:47:44.255273+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-09-10T18:47:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[18597]: audit 2024-09-10T18:47:44.258927+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[31607]: audit 2024-09-10T18:47:44.248964+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:45.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[31607]: audit 2024-09-10T18:47:44.255122+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:47:45.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[31607]: cluster 2024-09-10T18:47:44.255273+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-09-10T18:47:45.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:45 smithi100 bash[31607]: audit 2024-09-10T18:47:44.258927+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:45.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:45 smithi151 bash[22565]: audit 2024-09-10T18:47:44.248964+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:45.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:45 smithi151 bash[22565]: audit 2024-09-10T18:47:44.255122+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:47:45.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:45 smithi151 bash[22565]: cluster 2024-09-10T18:47:44.255273+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-09-10T18:47:45.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:45 smithi151 bash[22565]: audit 2024-09-10T18:47:44.258927+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]: dispatch 2024-09-10T18:47:46.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[31607]: audit 2024-09-10T18:47:45.265204+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]': finished 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[31607]: cluster 2024-09-10T18:47:45.265320+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[31607]: cluster 2024-09-10T18:47:46.071926+0000 mon.a (mon.0) 2826 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[31607]: cluster 2024-09-10T18:47:46.087621+0000 mgr.y (mgr.24512) 525 : cluster [DBG] pgmap v821: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[31607]: audit 2024-09-10T18:47:46.238045+0000 mon.c (mon.2) 471 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[18597]: audit 2024-09-10T18:47:45.265204+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]': finished 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[18597]: cluster 2024-09-10T18:47:45.265320+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[18597]: cluster 2024-09-10T18:47:46.071926+0000 mon.a (mon.0) 2826 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[18597]: cluster 2024-09-10T18:47:46.087621+0000 mgr.y (mgr.24512) 525 : cluster [DBG] pgmap v821: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:46.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:46 smithi100 bash[18597]: audit 2024-09-10T18:47:46.238045+0000 mon.c (mon.2) 471 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:46.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:46 smithi151 bash[22565]: audit 2024-09-10T18:47:45.265204+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-98"}]': finished 2024-09-10T18:47:46.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:46 smithi151 bash[22565]: cluster 2024-09-10T18:47:45.265320+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-09-10T18:47:46.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:46 smithi151 bash[22565]: cluster 2024-09-10T18:47:46.071926+0000 mon.a (mon.0) 2826 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:46.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:46 smithi151 bash[22565]: cluster 2024-09-10T18:47:46.087621+0000 mgr.y (mgr.24512) 525 : cluster [DBG] pgmap v821: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:46.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:46 smithi151 bash[22565]: audit 2024-09-10T18:47:46.238045+0000 mon.c (mon.2) 471 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[18597]: cluster 2024-09-10T18:47:46.280671+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-09-10T18:47:47.558 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[18597]: audit 2024-09-10T18:47:46.698476+0000 mon.c (mon.2) 472 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:47.558 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[18597]: audit 2024-09-10T18:47:46.700087+0000 mon.c (mon.2) 473 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:47.558 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[18597]: audit 2024-09-10T18:47:46.744149+0000 mon.a (mon.0) 2828 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:47:47.558 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[31607]: cluster 2024-09-10T18:47:46.280671+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-09-10T18:47:47.558 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[31607]: audit 2024-09-10T18:47:46.698476+0000 mon.c (mon.2) 472 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:47.558 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[31607]: audit 2024-09-10T18:47:46.700087+0000 mon.c (mon.2) 473 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:47.558 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:47 smithi100 bash[31607]: audit 2024-09-10T18:47:46.744149+0000 mon.a (mon.0) 2828 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:47:47.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:47 smithi151 bash[22565]: cluster 2024-09-10T18:47:46.280671+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-09-10T18:47:47.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:47 smithi151 bash[22565]: audit 2024-09-10T18:47:46.698476+0000 mon.c (mon.2) 472 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:47.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:47 smithi151 bash[22565]: audit 2024-09-10T18:47:46.700087+0000 mon.c (mon.2) 473 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:47.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:47 smithi151 bash[22565]: audit 2024-09-10T18:47:46.744149+0000 mon.a (mon.0) 2828 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:47:48.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:48 smithi151 bash[22565]: audit 2024-09-10T18:47:47.286829+0000 mon.b (mon.1) 377 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:48.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:48 smithi151 bash[22565]: cluster 2024-09-10T18:47:47.293251+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-09-10T18:47:48.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:48 smithi151 bash[22565]: audit 2024-09-10T18:47:47.296947+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:48.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:48 smithi151 bash[22565]: cluster 2024-09-10T18:47:48.088431+0000 mgr.y (mgr.24512) 526 : cluster [DBG] pgmap v824: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[18597]: audit 2024-09-10T18:47:47.286829+0000 mon.b (mon.1) 377 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[18597]: cluster 2024-09-10T18:47:47.293251+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-09-10T18:47:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[18597]: audit 2024-09-10T18:47:47.296947+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[18597]: cluster 2024-09-10T18:47:48.088431+0000 mgr.y (mgr.24512) 526 : cluster [DBG] pgmap v824: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[31607]: audit 2024-09-10T18:47:47.286829+0000 mon.b (mon.1) 377 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[31607]: cluster 2024-09-10T18:47:47.293251+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-09-10T18:47:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[31607]: audit 2024-09-10T18:47:47.296947+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:48 smithi100 bash[31607]: cluster 2024-09-10T18:47:48.088431+0000 mgr.y (mgr.24512) 526 : cluster [DBG] pgmap v824: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:49.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:49 smithi151 bash[22565]: audit 2024-09-10T18:47:48.302101+0000 mon.b (mon.1) 378 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:49.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:49 smithi151 bash[22565]: audit 2024-09-10T18:47:48.307602+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:49.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:49 smithi151 bash[22565]: cluster 2024-09-10T18:47:48.307844+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-09-10T18:47:49.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:49 smithi151 bash[22565]: audit 2024-09-10T18:47:48.316466+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[18597]: audit 2024-09-10T18:47:48.302101+0000 mon.b (mon.1) 378 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[18597]: audit 2024-09-10T18:47:48.307602+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[18597]: cluster 2024-09-10T18:47:48.307844+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[18597]: audit 2024-09-10T18:47:48.316466+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[31607]: audit 2024-09-10T18:47:48.302101+0000 mon.b (mon.1) 378 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[31607]: audit 2024-09-10T18:47:48.307602+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[31607]: cluster 2024-09-10T18:47:48.307844+0000 mon.a (mon.0) 2832 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-09-10T18:47:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:49 smithi100 bash[31607]: audit 2024-09-10T18:47:48.316466+0000 mon.a (mon.0) 2833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:47:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:50 smithi151 bash[22565]: audit 2024-09-10T18:47:49.302883+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:50 smithi151 bash[22565]: audit 2024-09-10T18:47:49.308530+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:47:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:50 smithi151 bash[22565]: cluster 2024-09-10T18:47:49.309095+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-09-10T18:47:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:50 smithi151 bash[22565]: audit 2024-09-10T18:47:49.319982+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:50.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:50 smithi151 bash[22565]: cluster 2024-09-10T18:47:50.089209+0000 mgr.y (mgr.24512) 527 : cluster [DBG] pgmap v827: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[18597]: audit 2024-09-10T18:47:49.302883+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[18597]: audit 2024-09-10T18:47:49.308530+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:47:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[18597]: cluster 2024-09-10T18:47:49.309095+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-09-10T18:47:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[18597]: audit 2024-09-10T18:47:49.319982+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[18597]: cluster 2024-09-10T18:47:50.089209+0000 mgr.y (mgr.24512) 527 : cluster [DBG] pgmap v827: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:50.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[31607]: audit 2024-09-10T18:47:49.302883+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:50.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[31607]: audit 2024-09-10T18:47:49.308530+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:47:50.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[31607]: cluster 2024-09-10T18:47:49.309095+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-09-10T18:47:50.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[31607]: audit 2024-09-10T18:47:49.319982+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:50.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:50 smithi100 bash[31607]: cluster 2024-09-10T18:47:50.089209+0000 mgr.y (mgr.24512) 527 : cluster [DBG] pgmap v827: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:51.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:51 smithi151 bash[22565]: audit 2024-09-10T18:47:50.325431+0000 mon.b (mon.1) 380 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]: dispatch 2024-09-10T18:47:51.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:51 smithi151 bash[22565]: audit 2024-09-10T18:47:50.330313+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]': finished 2024-09-10T18:47:51.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:51 smithi151 bash[22565]: cluster 2024-09-10T18:47:50.330424+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-09-10T18:47:51.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:51 smithi151 bash[22565]: audit 2024-09-10T18:47:50.335374+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]: dispatch 2024-09-10T18:47:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:51] "GET /metrics HTTP/1.1" 200 228598 "" "Prometheus/2.43.0" 2024-09-10T18:47:51.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[18597]: audit 2024-09-10T18:47:50.325431+0000 mon.b (mon.1) 380 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]: dispatch 2024-09-10T18:47:51.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[18597]: audit 2024-09-10T18:47:50.330313+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]': finished 2024-09-10T18:47:51.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[18597]: cluster 2024-09-10T18:47:50.330424+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-09-10T18:47:51.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[18597]: audit 2024-09-10T18:47:50.335374+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]: dispatch 2024-09-10T18:47:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[31607]: audit 2024-09-10T18:47:50.325431+0000 mon.b (mon.1) 380 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]: dispatch 2024-09-10T18:47:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[31607]: audit 2024-09-10T18:47:50.330313+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-6", "overlaypool": "test-rados-api-smithi100-72764-100"}]': finished 2024-09-10T18:47:51.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[31607]: cluster 2024-09-10T18:47:50.330424+0000 mon.a (mon.0) 2838 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-09-10T18:47:51.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:51 smithi100 bash[31607]: audit 2024-09-10T18:47:50.335374+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]: dispatch 2024-09-10T18:47:52.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: cluster 2024-09-10T18:47:51.326929+0000 mon.a (mon.0) 2840 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:47:52.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: audit 2024-09-10T18:47:51.328782+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:47:52.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: audit 2024-09-10T18:47:51.336139+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]': finished 2024-09-10T18:47:52.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: cluster 2024-09-10T18:47:51.336390+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-09-10T18:47:52.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: audit 2024-09-10T18:47:51.338550+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:47:52.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: cluster 2024-09-10T18:47:52.090062+0000 mgr.y (mgr.24512) 528 : cluster [DBG] pgmap v830: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:52.633 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: audit 2024-09-10T18:47:52.339180+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:47:52.634 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[22565]: cluster 2024-09-10T18:47:52.339508+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-09-10T18:47:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: cluster 2024-09-10T18:47:51.326929+0000 mon.a (mon.0) 2840 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:47:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: audit 2024-09-10T18:47:51.328782+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:47:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: audit 2024-09-10T18:47:51.336139+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]': finished 2024-09-10T18:47:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: cluster 2024-09-10T18:47:51.336390+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-09-10T18:47:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: audit 2024-09-10T18:47:51.338550+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:47:52.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: cluster 2024-09-10T18:47:52.090062+0000 mgr.y (mgr.24512) 528 : cluster [DBG] pgmap v830: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: audit 2024-09-10T18:47:52.339180+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[18597]: cluster 2024-09-10T18:47:52.339508+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: cluster 2024-09-10T18:47:51.326929+0000 mon.a (mon.0) 2840 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: audit 2024-09-10T18:47:51.328782+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: audit 2024-09-10T18:47:51.336139+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-100", "mode": "writeback"}]': finished 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: cluster 2024-09-10T18:47:51.336390+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: audit 2024-09-10T18:47:51.338550+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: cluster 2024-09-10T18:47:52.090062+0000 mgr.y (mgr.24512) 528 : cluster [DBG] pgmap v830: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: audit 2024-09-10T18:47:52.339180+0000 mon.a (mon.0) 2844 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:47:52.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:52 smithi100 bash[31607]: cluster 2024-09-10T18:47:52.339508+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-09-10T18:47:52.921 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:47:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:47:53.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:53 smithi151 bash[22565]: audit 2024-09-10T18:47:52.332932+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:53.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:53 smithi151 bash[22565]: audit 2024-09-10T18:47:52.349698+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:53.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:53 smithi151 bash[22565]: audit 2024-09-10T18:47:52.632133+0000 mgr.y (mgr.24512) 529 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:53 smithi100 bash[18597]: audit 2024-09-10T18:47:52.332932+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:53 smithi100 bash[18597]: audit 2024-09-10T18:47:52.349698+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:53 smithi100 bash[18597]: audit 2024-09-10T18:47:52.632133+0000 mgr.y (mgr.24512) 529 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:53 smithi100 bash[31607]: audit 2024-09-10T18:47:52.332932+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:53 smithi100 bash[31607]: audit 2024-09-10T18:47:52.349698+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:47:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:53 smithi100 bash[31607]: audit 2024-09-10T18:47:52.632133+0000 mgr.y (mgr.24512) 529 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:54.671 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:47:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:47:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:54.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:54 smithi151 bash[22565]: audit 2024-09-10T18:47:53.357615+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:47:54.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:54 smithi151 bash[22565]: audit 2024-09-10T18:47:53.361993+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:47:54.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:54 smithi151 bash[22565]: cluster 2024-09-10T18:47:53.362102+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-09-10T18:47:54.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:54 smithi151 bash[22565]: audit 2024-09-10T18:47:53.369768+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:47:54.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:54 smithi151 bash[22565]: cluster 2024-09-10T18:47:54.090874+0000 mgr.y (mgr.24512) 530 : cluster [DBG] pgmap v833: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:54.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:54 smithi151 bash[22565]: audit 2024-09-10T18:47:54.336534+0000 mon.c (mon.2) 474 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:54.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[18597]: audit 2024-09-10T18:47:53.357615+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:47:54.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[18597]: audit 2024-09-10T18:47:53.361993+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:47:54.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[18597]: cluster 2024-09-10T18:47:53.362102+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-09-10T18:47:54.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[18597]: audit 2024-09-10T18:47:53.369768+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:47:54.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[18597]: cluster 2024-09-10T18:47:54.090874+0000 mgr.y (mgr.24512) 530 : cluster [DBG] pgmap v833: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:54.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[18597]: audit 2024-09-10T18:47:54.336534+0000 mon.c (mon.2) 474 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:54.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[31607]: audit 2024-09-10T18:47:53.357615+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:47:54.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[31607]: audit 2024-09-10T18:47:53.361993+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:47:54.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[31607]: cluster 2024-09-10T18:47:53.362102+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-09-10T18:47:54.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[31607]: audit 2024-09-10T18:47:53.369768+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:47:54.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[31607]: cluster 2024-09-10T18:47:54.090874+0000 mgr.y (mgr.24512) 530 : cluster [DBG] pgmap v833: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:54.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:54 smithi100 bash[31607]: audit 2024-09-10T18:47:54.336534+0000 mon.c (mon.2) 474 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:55 smithi151 bash[22565]: cluster 2024-09-10T18:47:54.357455+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:47:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:55 smithi151 bash[22565]: audit 2024-09-10T18:47:54.362826+0000 mon.b (mon.1) 384 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:47:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:55 smithi151 bash[22565]: audit 2024-09-10T18:47:54.370748+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:47:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:55 smithi151 bash[22565]: cluster 2024-09-10T18:47:54.370877+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-09-10T18:47:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:55 smithi151 bash[22565]: audit 2024-09-10T18:47:54.373407+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:47:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:55 smithi151 bash[22565]: audit 2024-09-10T18:47:55.373926+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T18:47:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:55 smithi151 bash[22565]: cluster 2024-09-10T18:47:55.374055+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-09-10T18:47:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[18597]: cluster 2024-09-10T18:47:54.357455+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:47:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[18597]: audit 2024-09-10T18:47:54.362826+0000 mon.b (mon.1) 384 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:47:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[18597]: audit 2024-09-10T18:47:54.370748+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:47:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[18597]: cluster 2024-09-10T18:47:54.370877+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-09-10T18:47:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[18597]: audit 2024-09-10T18:47:54.373407+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:47:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[18597]: audit 2024-09-10T18:47:55.373926+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T18:47:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[18597]: cluster 2024-09-10T18:47:55.374055+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-09-10T18:47:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[31607]: cluster 2024-09-10T18:47:54.357455+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:47:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[31607]: audit 2024-09-10T18:47:54.362826+0000 mon.b (mon.1) 384 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:47:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[31607]: audit 2024-09-10T18:47:54.370748+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:47:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[31607]: cluster 2024-09-10T18:47:54.370877+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-09-10T18:47:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[31607]: audit 2024-09-10T18:47:54.373407+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:47:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[31607]: audit 2024-09-10T18:47:55.373926+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T18:47:55.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:55 smithi100 bash[31607]: cluster 2024-09-10T18:47:55.374055+0000 mon.a (mon.0) 2855 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-09-10T18:47:56.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:56 smithi151 bash[22565]: audit 2024-09-10T18:47:55.411146+0000 mon.b (mon.1) 385 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:56.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:56 smithi151 bash[22565]: audit 2024-09-10T18:47:55.420859+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:56.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:56 smithi151 bash[22565]: cluster 2024-09-10T18:47:56.091723+0000 mgr.y (mgr.24512) 531 : cluster [DBG] pgmap v836: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:56 smithi100 bash[18597]: audit 2024-09-10T18:47:55.411146+0000 mon.b (mon.1) 385 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:56 smithi100 bash[18597]: audit 2024-09-10T18:47:55.420859+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:56.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:56 smithi100 bash[18597]: cluster 2024-09-10T18:47:56.091723+0000 mgr.y (mgr.24512) 531 : cluster [DBG] pgmap v836: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:56.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:56 smithi100 bash[31607]: audit 2024-09-10T18:47:55.411146+0000 mon.b (mon.1) 385 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:56.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:56 smithi100 bash[31607]: audit 2024-09-10T18:47:55.420859+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:47:56.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:56 smithi100 bash[31607]: cluster 2024-09-10T18:47:56.091723+0000 mgr.y (mgr.24512) 531 : cluster [DBG] pgmap v836: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:57 smithi151 bash[22565]: audit 2024-09-10T18:47:56.388365+0000 mon.b (mon.1) 386 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:57.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:57 smithi151 bash[22565]: audit 2024-09-10T18:47:56.395961+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:47:57.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:57 smithi151 bash[22565]: cluster 2024-09-10T18:47:56.396185+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-09-10T18:47:57.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:57 smithi151 bash[22565]: audit 2024-09-10T18:47:56.399064+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[18597]: audit 2024-09-10T18:47:56.388365+0000 mon.b (mon.1) 386 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[18597]: audit 2024-09-10T18:47:56.395961+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[18597]: cluster 2024-09-10T18:47:56.396185+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[18597]: audit 2024-09-10T18:47:56.399064+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[31607]: audit 2024-09-10T18:47:56.388365+0000 mon.b (mon.1) 386 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[31607]: audit 2024-09-10T18:47:56.395961+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]': finished 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[31607]: cluster 2024-09-10T18:47:56.396185+0000 mon.a (mon.0) 2858 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-09-10T18:47:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:57 smithi100 bash[31607]: audit 2024-09-10T18:47:56.399064+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]: dispatch 2024-09-10T18:47:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:58 smithi151 bash[22565]: audit 2024-09-10T18:47:57.399802+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]': finished 2024-09-10T18:47:58.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:58 smithi151 bash[22565]: cluster 2024-09-10T18:47:57.400001+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-09-10T18:47:58.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:58 smithi151 bash[22565]: cluster 2024-09-10T18:47:58.092536+0000 mgr.y (mgr.24512) 532 : cluster [DBG] pgmap v839: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:58.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:58 smithi100 bash[31607]: audit 2024-09-10T18:47:57.399802+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]': finished 2024-09-10T18:47:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:58 smithi100 bash[31607]: cluster 2024-09-10T18:47:57.400001+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-09-10T18:47:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:58 smithi100 bash[31607]: cluster 2024-09-10T18:47:58.092536+0000 mgr.y (mgr.24512) 532 : cluster [DBG] pgmap v839: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:58.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:58 smithi100 bash[18597]: audit 2024-09-10T18:47:57.399802+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-100"}]': finished 2024-09-10T18:47:58.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:58 smithi100 bash[18597]: cluster 2024-09-10T18:47:57.400001+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-09-10T18:47:58.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:58 smithi100 bash[18597]: cluster 2024-09-10T18:47:58.092536+0000 mgr.y (mgr.24512) 532 : cluster [DBG] pgmap v839: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:47:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:47:59 smithi151 bash[22565]: cluster 2024-09-10T18:47:58.419531+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-09-10T18:47:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:47:59 smithi100 bash[18597]: cluster 2024-09-10T18:47:58.419531+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-09-10T18:47:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:47:59 smithi100 bash[31607]: cluster 2024-09-10T18:47:58.419531+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-09-10T18:48:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[18597]: audit 2024-09-10T18:47:59.429775+0000 mon.b (mon.1) 387 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[18597]: cluster 2024-09-10T18:47:59.432589+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-09-10T18:48:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[18597]: audit 2024-09-10T18:47:59.450307+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[18597]: cluster 2024-09-10T18:48:00.093355+0000 mgr.y (mgr.24512) 533 : cluster [DBG] pgmap v842: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[31607]: audit 2024-09-10T18:47:59.429775+0000 mon.b (mon.1) 387 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[31607]: cluster 2024-09-10T18:47:59.432589+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-09-10T18:48:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[31607]: audit 2024-09-10T18:47:59.450307+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:00.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:00 smithi100 bash[31607]: cluster 2024-09-10T18:48:00.093355+0000 mgr.y (mgr.24512) 533 : cluster [DBG] pgmap v842: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:00.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:00 smithi151 bash[22565]: audit 2024-09-10T18:47:59.429775+0000 mon.b (mon.1) 387 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:00.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:00 smithi151 bash[22565]: cluster 2024-09-10T18:47:59.432589+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-09-10T18:48:00.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:00 smithi151 bash[22565]: audit 2024-09-10T18:47:59.450307+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:00.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:00 smithi151 bash[22565]: cluster 2024-09-10T18:48:00.093355+0000 mgr.y (mgr.24512) 533 : cluster [DBG] pgmap v842: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:01] "GET /metrics HTTP/1.1" 200 228657 "" "Prometheus/2.43.0" 2024-09-10T18:48:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[18597]: audit 2024-09-10T18:48:00.434506+0000 mon.b (mon.1) 388 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[18597]: audit 2024-09-10T18:48:00.439429+0000 mon.b (mon.1) 389 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[18597]: audit 2024-09-10T18:48:00.441057+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[18597]: cluster 2024-09-10T18:48:00.441460+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-09-10T18:48:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[18597]: cluster 2024-09-10T18:48:00.448262+0000 mon.a (mon.0) 2867 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[18597]: audit 2024-09-10T18:48:00.465645+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[31607]: audit 2024-09-10T18:48:00.434506+0000 mon.b (mon.1) 388 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:01.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[31607]: audit 2024-09-10T18:48:00.439429+0000 mon.b (mon.1) 389 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:01.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[31607]: audit 2024-09-10T18:48:00.441057+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:01.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[31607]: cluster 2024-09-10T18:48:00.441460+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-09-10T18:48:01.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[31607]: cluster 2024-09-10T18:48:00.448262+0000 mon.a (mon.0) 2867 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:01.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:01 smithi100 bash[31607]: audit 2024-09-10T18:48:00.465645+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:01.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:01 smithi151 bash[22565]: audit 2024-09-10T18:48:00.434506+0000 mon.b (mon.1) 388 : audit [DBG] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:01.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:01 smithi151 bash[22565]: audit 2024-09-10T18:48:00.439429+0000 mon.b (mon.1) 389 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:01.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:01 smithi151 bash[22565]: audit 2024-09-10T18:48:00.441057+0000 mon.a (mon.0) 2865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:01.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:01 smithi151 bash[22565]: cluster 2024-09-10T18:48:00.441460+0000 mon.a (mon.0) 2866 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-09-10T18:48:01.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:01 smithi151 bash[22565]: cluster 2024-09-10T18:48:00.448262+0000 mon.a (mon.0) 2867 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:01.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:01 smithi151 bash[22565]: audit 2024-09-10T18:48:00.465645+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[18597]: audit 2024-09-10T18:48:01.455529+0000 mon.b (mon.1) 390 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[18597]: audit 2024-09-10T18:48:01.462881+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[18597]: cluster 2024-09-10T18:48:01.463106+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[18597]: audit 2024-09-10T18:48:01.473082+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[18597]: cluster 2024-09-10T18:48:02.094218+0000 mgr.y (mgr.24512) 534 : cluster [DBG] pgmap v845: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[31607]: audit 2024-09-10T18:48:01.455529+0000 mon.b (mon.1) 390 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[31607]: audit 2024-09-10T18:48:01.462881+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[31607]: cluster 2024-09-10T18:48:01.463106+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-09-10T18:48:02.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[31607]: audit 2024-09-10T18:48:01.473082+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:02.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:02 smithi100 bash[31607]: cluster 2024-09-10T18:48:02.094218+0000 mgr.y (mgr.24512) 534 : cluster [DBG] pgmap v845: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:02.921 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:48:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:48:02.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:02 smithi151 bash[22565]: audit 2024-09-10T18:48:01.455529+0000 mon.b (mon.1) 390 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:02.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:02 smithi151 bash[22565]: audit 2024-09-10T18:48:01.462881+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:02.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:02 smithi151 bash[22565]: cluster 2024-09-10T18:48:01.463106+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-09-10T18:48:02.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:02 smithi151 bash[22565]: audit 2024-09-10T18:48:01.473082+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:02.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:02 smithi151 bash[22565]: cluster 2024-09-10T18:48:02.094218+0000 mgr.y (mgr.24512) 534 : cluster [DBG] pgmap v845: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[18597]: audit 2024-09-10T18:48:02.477517+0000 mon.b (mon.1) 391 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[18597]: audit 2024-09-10T18:48:02.485323+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[18597]: cluster 2024-09-10T18:48:02.485487+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[18597]: audit 2024-09-10T18:48:02.488294+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[18597]: audit 2024-09-10T18:48:02.643554+0000 mgr.y (mgr.24512) 535 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[31607]: audit 2024-09-10T18:48:02.477517+0000 mon.b (mon.1) 391 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[31607]: audit 2024-09-10T18:48:02.485323+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[31607]: cluster 2024-09-10T18:48:02.485487+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-09-10T18:48:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[31607]: audit 2024-09-10T18:48:02.488294+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:03 smithi100 bash[31607]: audit 2024-09-10T18:48:02.643554+0000 mgr.y (mgr.24512) 535 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:03.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:03 smithi151 bash[22565]: audit 2024-09-10T18:48:02.477517+0000 mon.b (mon.1) 391 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:03.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:03 smithi151 bash[22565]: audit 2024-09-10T18:48:02.485323+0000 mon.a (mon.0) 2872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:03.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:03 smithi151 bash[22565]: cluster 2024-09-10T18:48:02.485487+0000 mon.a (mon.0) 2873 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-09-10T18:48:03.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:03 smithi151 bash[22565]: audit 2024-09-10T18:48:02.488294+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:03.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:03 smithi151 bash[22565]: audit 2024-09-10T18:48:02.643554+0000 mgr.y (mgr.24512) 535 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[18597]: audit 2024-09-10T18:48:03.488550+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[18597]: cluster 2024-09-10T18:48:03.488742+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-09-10T18:48:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[18597]: audit 2024-09-10T18:48:03.517504+0000 mon.b (mon.1) 392 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:48:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[18597]: audit 2024-09-10T18:48:03.519030+0000 mon.b (mon.1) 393 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-102"}]: dispatch 2024-09-10T18:48:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[18597]: audit 2024-09-10T18:48:03.527125+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:48:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[18597]: audit 2024-09-10T18:48:03.528538+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-102"}]: dispatch 2024-09-10T18:48:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[18597]: cluster 2024-09-10T18:48:04.095107+0000 mgr.y (mgr.24512) 536 : cluster [DBG] pgmap v848: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:04.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[31607]: audit 2024-09-10T18:48:03.488550+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:04.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[31607]: cluster 2024-09-10T18:48:03.488742+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-09-10T18:48:04.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[31607]: audit 2024-09-10T18:48:03.517504+0000 mon.b (mon.1) 392 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:48:04.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[31607]: audit 2024-09-10T18:48:03.519030+0000 mon.b (mon.1) 393 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-102"}]: dispatch 2024-09-10T18:48:04.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[31607]: audit 2024-09-10T18:48:03.527125+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:48:04.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[31607]: audit 2024-09-10T18:48:03.528538+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-102"}]: dispatch 2024-09-10T18:48:04.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:04 smithi100 bash[31607]: cluster 2024-09-10T18:48:04.095107+0000 mgr.y (mgr.24512) 536 : cluster [DBG] pgmap v848: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:04.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[22565]: audit 2024-09-10T18:48:03.488550+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:04.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[22565]: cluster 2024-09-10T18:48:03.488742+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-09-10T18:48:04.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[22565]: audit 2024-09-10T18:48:03.517504+0000 mon.b (mon.1) 392 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:48:04.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[22565]: audit 2024-09-10T18:48:03.519030+0000 mon.b (mon.1) 393 : audit [INF] from='client.? 172.21.15.100:0/2392037152' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-102"}]: dispatch 2024-09-10T18:48:04.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[22565]: audit 2024-09-10T18:48:03.527125+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-6"}]: dispatch 2024-09-10T18:48:04.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[22565]: audit 2024-09-10T18:48:03.528538+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-6", "tierpool": "test-rados-api-smithi100-72764-102"}]: dispatch 2024-09-10T18:48:04.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[22565]: cluster 2024-09-10T18:48:04.095107+0000 mgr.y (mgr.24512) 536 : cluster [DBG] pgmap v848: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:04.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:48:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:05.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:05 smithi100 bash[18597]: cluster 2024-09-10T18:48:04.510047+0000 mon.a (mon.0) 2879 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-09-10T18:48:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:05 smithi100 bash[31607]: cluster 2024-09-10T18:48:04.510047+0000 mon.a (mon.0) 2879 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-09-10T18:48:05.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:05 smithi151 bash[22565]: cluster 2024-09-10T18:48:04.510047+0000 mon.a (mon.0) 2879 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: cluster 2024-09-10T18:48:05.525155+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: audit 2024-09-10T18:48:05.550684+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: audit 2024-09-10T18:48:05.551089+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: audit 2024-09-10T18:48:05.551933+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: audit 2024-09-10T18:48:05.552291+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: audit 2024-09-10T18:48:05.552927+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: audit 2024-09-10T18:48:05.553300+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: cluster 2024-09-10T18:48:06.076807+0000 mon.a (mon.0) 2884 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[18597]: cluster 2024-09-10T18:48:06.095863+0000 mgr.y (mgr.24512) 537 : cluster [DBG] pgmap v851: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: cluster 2024-09-10T18:48:05.525155+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: audit 2024-09-10T18:48:05.550684+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: audit 2024-09-10T18:48:05.551089+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: audit 2024-09-10T18:48:05.551933+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: audit 2024-09-10T18:48:05.552291+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: audit 2024-09-10T18:48:05.552927+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: audit 2024-09-10T18:48:05.553300+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: cluster 2024-09-10T18:48:06.076807+0000 mon.a (mon.0) 2884 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:06.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:06 smithi100 bash[31607]: cluster 2024-09-10T18:48:06.095863+0000 mgr.y (mgr.24512) 537 : cluster [DBG] pgmap v851: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:06.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: cluster 2024-09-10T18:48:05.525155+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: audit 2024-09-10T18:48:05.550684+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: audit 2024-09-10T18:48:05.551089+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: audit 2024-09-10T18:48:05.551933+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: audit 2024-09-10T18:48:05.552291+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: audit 2024-09-10T18:48:05.552927+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: audit 2024-09-10T18:48:05.553300+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: cluster 2024-09-10T18:48:06.076807+0000 mon.a (mon.0) 2884 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:06 smithi151 bash[22565]: cluster 2024-09-10T18:48:06.095863+0000 mgr.y (mgr.24512) 537 : cluster [DBG] pgmap v851: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[18597]: audit 2024-09-10T18:48:06.534316+0000 mon.a (mon.0) 2885 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[18597]: cluster 2024-09-10T18:48:06.534435+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-09-10T18:48:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[18597]: audit 2024-09-10T18:48:06.535754+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:08.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[18597]: audit 2024-09-10T18:48:06.538097+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:08.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[31607]: audit 2024-09-10T18:48:06.534316+0000 mon.a (mon.0) 2885 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[31607]: cluster 2024-09-10T18:48:06.534435+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-09-10T18:48:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[31607]: audit 2024-09-10T18:48:06.535754+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:08.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:07 smithi100 bash[31607]: audit 2024-09-10T18:48:06.538097+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:08.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:07 smithi151 bash[22565]: audit 2024-09-10T18:48:06.534316+0000 mon.a (mon.0) 2885 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:08.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:07 smithi151 bash[22565]: cluster 2024-09-10T18:48:06.534435+0000 mon.a (mon.0) 2886 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-09-10T18:48:08.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:07 smithi151 bash[22565]: audit 2024-09-10T18:48:06.535754+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:08.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:07 smithi151 bash[22565]: audit 2024-09-10T18:48:06.538097+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:09.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[18597]: cluster 2024-09-10T18:48:07.964430+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-09-10T18:48:09.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[18597]: cluster 2024-09-10T18:48:08.096606+0000 mgr.y (mgr.24512) 538 : cluster [DBG] pgmap v854: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:09.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[18597]: audit 2024-09-10T18:48:08.542465+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:09.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[18597]: cluster 2024-09-10T18:48:08.542834+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-09-10T18:48:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[31607]: cluster 2024-09-10T18:48:07.964430+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-09-10T18:48:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[31607]: cluster 2024-09-10T18:48:08.096606+0000 mgr.y (mgr.24512) 538 : cluster [DBG] pgmap v854: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:09.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[31607]: audit 2024-09-10T18:48:08.542465+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:09.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:08 smithi100 bash[31607]: cluster 2024-09-10T18:48:08.542834+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-09-10T18:48:09.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:08 smithi151 bash[22565]: cluster 2024-09-10T18:48:07.964430+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-09-10T18:48:09.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:08 smithi151 bash[22565]: cluster 2024-09-10T18:48:08.096606+0000 mgr.y (mgr.24512) 538 : cluster [DBG] pgmap v854: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:09.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:08 smithi151 bash[22565]: audit 2024-09-10T18:48:08.542465+0000 mon.a (mon.0) 2889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-72764-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:09.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:08 smithi151 bash[22565]: cluster 2024-09-10T18:48:08.542834+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-09-10T18:48:10.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:10 smithi151 bash[22565]: audit 2024-09-10T18:48:09.344635+0000 mon.a (mon.0) 2891 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:48:10.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:10 smithi151 bash[22565]: audit 2024-09-10T18:48:09.347334+0000 mon.c (mon.2) 479 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:10.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:10 smithi151 bash[22565]: cluster 2024-09-10T18:48:09.545764+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-09-10T18:48:10.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:10 smithi151 bash[22565]: cluster 2024-09-10T18:48:10.097268+0000 mgr.y (mgr.24512) 539 : cluster [DBG] pgmap v857: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[18597]: audit 2024-09-10T18:48:09.344635+0000 mon.a (mon.0) 2891 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[18597]: audit 2024-09-10T18:48:09.347334+0000 mon.c (mon.2) 479 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[18597]: cluster 2024-09-10T18:48:09.545764+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[18597]: cluster 2024-09-10T18:48:10.097268+0000 mgr.y (mgr.24512) 539 : cluster [DBG] pgmap v857: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[31607]: audit 2024-09-10T18:48:09.344635+0000 mon.a (mon.0) 2891 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[31607]: audit 2024-09-10T18:48:09.347334+0000 mon.c (mon.2) 479 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[31607]: cluster 2024-09-10T18:48:09.545764+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-09-10T18:48:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:10 smithi100 bash[31607]: cluster 2024-09-10T18:48:10.097268+0000 mgr.y (mgr.24512) 539 : cluster [DBG] pgmap v857: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:11.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:48:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:11] "GET /metrics HTTP/1.1" 200 226245 "" "Prometheus/2.43.0" 2024-09-10T18:48:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:11 smithi100 bash[31607]: cluster 2024-09-10T18:48:10.555232+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-09-10T18:48:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:11 smithi100 bash[31607]: audit 2024-09-10T18:48:10.563052+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:11 smithi100 bash[31607]: cluster 2024-09-10T18:48:11.077954+0000 mon.a (mon.0) 2895 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:11 smithi100 bash[18597]: cluster 2024-09-10T18:48:10.555232+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-09-10T18:48:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:11 smithi100 bash[18597]: audit 2024-09-10T18:48:10.563052+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:11.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:11 smithi100 bash[18597]: cluster 2024-09-10T18:48:11.077954+0000 mon.a (mon.0) 2895 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:11.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:11 smithi151 bash[22565]: cluster 2024-09-10T18:48:10.555232+0000 mon.a (mon.0) 2893 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-09-10T18:48:11.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:11 smithi151 bash[22565]: audit 2024-09-10T18:48:10.563052+0000 mon.a (mon.0) 2894 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:11.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:11 smithi151 bash[22565]: cluster 2024-09-10T18:48:11.077954+0000 mon.a (mon.0) 2895 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:12.921 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:48:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:48:12.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:12 smithi151 bash[22565]: audit 2024-09-10T18:48:11.558866+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:12.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:12 smithi151 bash[22565]: cluster 2024-09-10T18:48:11.558973+0000 mon.a (mon.0) 2897 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-09-10T18:48:12.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:12 smithi151 bash[22565]: cluster 2024-09-10T18:48:12.098024+0000 mgr.y (mgr.24512) 540 : cluster [DBG] pgmap v860: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:13.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:12 smithi100 bash[18597]: audit 2024-09-10T18:48:11.558866+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:12 smithi100 bash[18597]: cluster 2024-09-10T18:48:11.558973+0000 mon.a (mon.0) 2897 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-09-10T18:48:13.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:12 smithi100 bash[18597]: cluster 2024-09-10T18:48:12.098024+0000 mgr.y (mgr.24512) 540 : cluster [DBG] pgmap v860: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:12 smithi100 bash[31607]: audit 2024-09-10T18:48:11.558866+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:12 smithi100 bash[31607]: cluster 2024-09-10T18:48:11.558973+0000 mon.a (mon.0) 2897 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-09-10T18:48:13.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:12 smithi100 bash[31607]: cluster 2024-09-10T18:48:12.098024+0000 mgr.y (mgr.24512) 540 : cluster [DBG] pgmap v860: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:13.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:13 smithi151 bash[22565]: cluster 2024-09-10T18:48:12.581183+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-09-10T18:48:13.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:13 smithi151 bash[22565]: audit 2024-09-10T18:48:12.582952+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:13.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:13 smithi151 bash[22565]: audit 2024-09-10T18:48:12.648085+0000 mgr.y (mgr.24512) 541 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:14.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:13 smithi100 bash[18597]: cluster 2024-09-10T18:48:12.581183+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-09-10T18:48:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:13 smithi100 bash[18597]: audit 2024-09-10T18:48:12.582952+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:14.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:13 smithi100 bash[18597]: audit 2024-09-10T18:48:12.648085+0000 mgr.y (mgr.24512) 541 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:13 smithi100 bash[31607]: cluster 2024-09-10T18:48:12.581183+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-09-10T18:48:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:13 smithi100 bash[31607]: audit 2024-09-10T18:48:12.582952+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:14.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:13 smithi100 bash[31607]: audit 2024-09-10T18:48:12.648085+0000 mgr.y (mgr.24512) 541 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:14.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:48:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:14.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:14 smithi151 bash[22565]: audit 2024-09-10T18:48:13.590151+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:14.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:14 smithi151 bash[22565]: cluster 2024-09-10T18:48:13.590826+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-09-10T18:48:14.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:14 smithi151 bash[22565]: audit 2024-09-10T18:48:13.609955+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:14.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:14 smithi151 bash[22565]: cluster 2024-09-10T18:48:14.098980+0000 mgr.y (mgr.24512) 542 : cluster [DBG] pgmap v863: 236 pgs: 64 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[18597]: audit 2024-09-10T18:48:13.590151+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[18597]: cluster 2024-09-10T18:48:13.590826+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-09-10T18:48:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[18597]: audit 2024-09-10T18:48:13.609955+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:15.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[18597]: cluster 2024-09-10T18:48:14.098980+0000 mgr.y (mgr.24512) 542 : cluster [DBG] pgmap v863: 236 pgs: 64 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[31607]: audit 2024-09-10T18:48:13.590151+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:15.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[31607]: cluster 2024-09-10T18:48:13.590826+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-09-10T18:48:15.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[31607]: audit 2024-09-10T18:48:13.609955+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:15.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:14 smithi100 bash[31607]: cluster 2024-09-10T18:48:14.098980+0000 mgr.y (mgr.24512) 542 : cluster [DBG] pgmap v863: 236 pgs: 64 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:48:15.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:15 smithi151 bash[22565]: audit 2024-09-10T18:48:14.611073+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:15.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:15 smithi151 bash[22565]: cluster 2024-09-10T18:48:14.611280+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-09-10T18:48:15.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:15 smithi151 bash[22565]: audit 2024-09-10T18:48:14.612136+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-107", "overlaypool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:16.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:15 smithi100 bash[18597]: audit 2024-09-10T18:48:14.611073+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:16.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:15 smithi100 bash[18597]: cluster 2024-09-10T18:48:14.611280+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-09-10T18:48:16.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:15 smithi100 bash[18597]: audit 2024-09-10T18:48:14.612136+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-107", "overlaypool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:16.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:15 smithi100 bash[31607]: audit 2024-09-10T18:48:14.611073+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:16.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:15 smithi100 bash[31607]: cluster 2024-09-10T18:48:14.611280+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-09-10T18:48:16.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:15 smithi100 bash[31607]: audit 2024-09-10T18:48:14.612136+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-107", "overlaypool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:16.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:16 smithi151 bash[22565]: audit 2024-09-10T18:48:15.613306+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-107", "overlaypool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:16.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:16 smithi151 bash[22565]: cluster 2024-09-10T18:48:15.613431+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-09-10T18:48:16.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:16 smithi151 bash[22565]: audit 2024-09-10T18:48:15.615533+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:16.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:16 smithi151 bash[22565]: cluster 2024-09-10T18:48:16.099955+0000 mgr.y (mgr.24512) 543 : cluster [DBG] pgmap v866: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:16 smithi151 bash[22565]: cluster 2024-09-10T18:48:16.607003+0000 mon.a (mon.0) 2909 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:16 smithi151 bash[22565]: audit 2024-09-10T18:48:16.615644+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-107-cache", "mode": "writeback"}]': finished 2024-09-10T18:48:16.923 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:16 smithi151 bash[22565]: cluster 2024-09-10T18:48:16.615768+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-09-10T18:48:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[18597]: audit 2024-09-10T18:48:15.613306+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-107", "overlaypool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[18597]: cluster 2024-09-10T18:48:15.613431+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-09-10T18:48:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[18597]: audit 2024-09-10T18:48:15.615533+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[18597]: cluster 2024-09-10T18:48:16.099955+0000 mgr.y (mgr.24512) 543 : cluster [DBG] pgmap v866: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[18597]: cluster 2024-09-10T18:48:16.607003+0000 mon.a (mon.0) 2909 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[18597]: audit 2024-09-10T18:48:16.615644+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-107-cache", "mode": "writeback"}]': finished 2024-09-10T18:48:17.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[18597]: cluster 2024-09-10T18:48:16.615768+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-09-10T18:48:17.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[31607]: audit 2024-09-10T18:48:15.613306+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-107", "overlaypool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:17.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[31607]: cluster 2024-09-10T18:48:15.613431+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-09-10T18:48:17.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[31607]: audit 2024-09-10T18:48:15.615533+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:17.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[31607]: cluster 2024-09-10T18:48:16.099955+0000 mgr.y (mgr.24512) 543 : cluster [DBG] pgmap v866: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:17.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[31607]: cluster 2024-09-10T18:48:16.607003+0000 mon.a (mon.0) 2909 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:17.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[31607]: audit 2024-09-10T18:48:16.615644+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-107-cache", "mode": "writeback"}]': finished 2024-09-10T18:48:17.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:16 smithi100 bash[31607]: cluster 2024-09-10T18:48:16.615768+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-09-10T18:48:17.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:17 smithi151 bash[22565]: audit 2024-09-10T18:48:16.642777+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-107"}]: dispatch 2024-09-10T18:48:18.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:17 smithi100 bash[18597]: audit 2024-09-10T18:48:16.642777+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-107"}]: dispatch 2024-09-10T18:48:18.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:17 smithi100 bash[31607]: audit 2024-09-10T18:48:16.642777+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-107"}]: dispatch 2024-09-10T18:48:18.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:18 smithi151 bash[22565]: audit 2024-09-10T18:48:17.651274+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-107"}]': finished 2024-09-10T18:48:18.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:18 smithi151 bash[22565]: cluster 2024-09-10T18:48:17.651377+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-09-10T18:48:18.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:18 smithi151 bash[22565]: audit 2024-09-10T18:48:17.653565+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:18.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:18 smithi151 bash[22565]: cluster 2024-09-10T18:48:18.100841+0000 mgr.y (mgr.24512) 544 : cluster [DBG] pgmap v869: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:19.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[31607]: audit 2024-09-10T18:48:17.651274+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-107"}]': finished 2024-09-10T18:48:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[31607]: cluster 2024-09-10T18:48:17.651377+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-09-10T18:48:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[31607]: audit 2024-09-10T18:48:17.653565+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:19.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[31607]: cluster 2024-09-10T18:48:18.100841+0000 mgr.y (mgr.24512) 544 : cluster [DBG] pgmap v869: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[18597]: audit 2024-09-10T18:48:17.651274+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-107"}]': finished 2024-09-10T18:48:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[18597]: cluster 2024-09-10T18:48:17.651377+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-09-10T18:48:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[18597]: audit 2024-09-10T18:48:17.653565+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]: dispatch 2024-09-10T18:48:19.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:18 smithi100 bash[18597]: cluster 2024-09-10T18:48:18.100841+0000 mgr.y (mgr.24512) 544 : cluster [DBG] pgmap v869: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:19.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:19 smithi151 bash[22565]: cluster 2024-09-10T18:48:18.640665+0000 mon.a (mon.0) 2916 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:19.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:19 smithi151 bash[22565]: audit 2024-09-10T18:48:18.649731+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:19.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:19 smithi151 bash[22565]: cluster 2024-09-10T18:48:18.649889+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-09-10T18:48:20.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:19 smithi100 bash[18597]: cluster 2024-09-10T18:48:18.640665+0000 mon.a (mon.0) 2916 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:19 smithi100 bash[18597]: audit 2024-09-10T18:48:18.649731+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:19 smithi100 bash[18597]: cluster 2024-09-10T18:48:18.649889+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-09-10T18:48:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:19 smithi100 bash[31607]: cluster 2024-09-10T18:48:18.640665+0000 mon.a (mon.0) 2916 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:19 smithi100 bash[31607]: audit 2024-09-10T18:48:18.649731+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? 172.21.15.100:0/688759327' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-107", "tierpool": "test-rados-api-smithi100-72764-107-cache"}]': finished 2024-09-10T18:48:20.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:19 smithi100 bash[31607]: cluster 2024-09-10T18:48:18.649889+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-09-10T18:48:21.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:20 smithi100 bash[31607]: cluster 2024-09-10T18:48:19.662566+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-09-10T18:48:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:20 smithi100 bash[31607]: cluster 2024-09-10T18:48:20.101721+0000 mgr.y (mgr.24512) 545 : cluster [DBG] pgmap v872: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:21.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:20 smithi100 bash[31607]: cluster 2024-09-10T18:48:20.666282+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-09-10T18:48:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:20 smithi100 bash[18597]: cluster 2024-09-10T18:48:19.662566+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-09-10T18:48:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:20 smithi100 bash[18597]: cluster 2024-09-10T18:48:20.101721+0000 mgr.y (mgr.24512) 545 : cluster [DBG] pgmap v872: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:20 smithi100 bash[18597]: cluster 2024-09-10T18:48:20.666282+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-09-10T18:48:21.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:20 smithi151 bash[22565]: cluster 2024-09-10T18:48:19.662566+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-09-10T18:48:21.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:20 smithi151 bash[22565]: cluster 2024-09-10T18:48:20.101721+0000 mgr.y (mgr.24512) 545 : cluster [DBG] pgmap v872: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:21.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:20 smithi151 bash[22565]: cluster 2024-09-10T18:48:20.666282+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-09-10T18:48:21.685 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:21] "GET /metrics HTTP/1.1" 200 226245 "" "Prometheus/2.43.0" 2024-09-10T18:48:22.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[31607]: audit 2024-09-10T18:48:20.693988+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[31607]: audit 2024-09-10T18:48:20.694269+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[31607]: audit 2024-09-10T18:48:20.695249+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[31607]: audit 2024-09-10T18:48:20.695524+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[31607]: audit 2024-09-10T18:48:20.695985+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:22.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[31607]: audit 2024-09-10T18:48:20.696234+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:22.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[18597]: audit 2024-09-10T18:48:20.693988+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[18597]: audit 2024-09-10T18:48:20.694269+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[18597]: audit 2024-09-10T18:48:20.695249+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[18597]: audit 2024-09-10T18:48:20.695524+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[18597]: audit 2024-09-10T18:48:20.695985+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:22.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:21 smithi100 bash[18597]: audit 2024-09-10T18:48:20.696234+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:22.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:21 smithi151 bash[22565]: audit 2024-09-10T18:48:20.693988+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:21 smithi151 bash[22565]: audit 2024-09-10T18:48:20.694269+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:21 smithi151 bash[22565]: audit 2024-09-10T18:48:20.695249+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:21 smithi151 bash[22565]: audit 2024-09-10T18:48:20.695524+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:22.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:21 smithi151 bash[22565]: audit 2024-09-10T18:48:20.695985+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:22.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:21 smithi151 bash[22565]: audit 2024-09-10T18:48:20.696234+0000 mon.a (mon.0) 2923 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:22.921 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:48:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:48:23.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:23 smithi151 bash[22565]: audit 2024-09-10T18:48:21.699352+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:23.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:23 smithi151 bash[22565]: cluster 2024-09-10T18:48:21.699571+0000 mon.a (mon.0) 2925 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-09-10T18:48:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:23 smithi151 bash[22565]: audit 2024-09-10T18:48:21.703681+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:23 smithi151 bash[22565]: audit 2024-09-10T18:48:21.704281+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:23.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:23 smithi151 bash[22565]: cluster 2024-09-10T18:48:22.102549+0000 mgr.y (mgr.24512) 546 : cluster [DBG] pgmap v875: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[18597]: audit 2024-09-10T18:48:21.699352+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[18597]: cluster 2024-09-10T18:48:21.699571+0000 mon.a (mon.0) 2925 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-09-10T18:48:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[18597]: audit 2024-09-10T18:48:21.703681+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[18597]: audit 2024-09-10T18:48:21.704281+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[18597]: cluster 2024-09-10T18:48:22.102549+0000 mgr.y (mgr.24512) 546 : cluster [DBG] pgmap v875: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[31607]: audit 2024-09-10T18:48:21.699352+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[31607]: cluster 2024-09-10T18:48:21.699571+0000 mon.a (mon.0) 2925 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-09-10T18:48:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[31607]: audit 2024-09-10T18:48:21.703681+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[31607]: audit 2024-09-10T18:48:21.704281+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:23 smithi100 bash[31607]: cluster 2024-09-10T18:48:22.102549+0000 mgr.y (mgr.24512) 546 : cluster [DBG] pgmap v875: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:24.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:24 smithi151 bash[22565]: audit 2024-09-10T18:48:22.659427+0000 mgr.y (mgr.24512) 547 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:24.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:24 smithi151 bash[22565]: cluster 2024-09-10T18:48:23.136727+0000 mon.a (mon.0) 2927 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-09-10T18:48:24.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:24 smithi151 bash[22565]: audit 2024-09-10T18:48:23.710678+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:24.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:24 smithi151 bash[22565]: cluster 2024-09-10T18:48:23.711052+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[18597]: audit 2024-09-10T18:48:22.659427+0000 mgr.y (mgr.24512) 547 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[18597]: cluster 2024-09-10T18:48:23.136727+0000 mon.a (mon.0) 2927 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[18597]: audit 2024-09-10T18:48:23.710678+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[18597]: cluster 2024-09-10T18:48:23.711052+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[31607]: audit 2024-09-10T18:48:22.659427+0000 mgr.y (mgr.24512) 547 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[31607]: cluster 2024-09-10T18:48:23.136727+0000 mon.a (mon.0) 2927 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[31607]: audit 2024-09-10T18:48:23.710678+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:24 smithi100 bash[31607]: cluster 2024-09-10T18:48:23.711052+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-09-10T18:48:24.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:48:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:25.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:25 smithi151 bash[22565]: cluster 2024-09-10T18:48:24.103291+0000 mgr.y (mgr.24512) 548 : cluster [DBG] pgmap v878: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:25.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:25 smithi151 bash[22565]: audit 2024-09-10T18:48:24.337155+0000 mon.c (mon.2) 484 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:25.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:25 smithi151 bash[22565]: cluster 2024-09-10T18:48:24.700156+0000 mon.a (mon.0) 2930 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:25.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:25 smithi151 bash[22565]: cluster 2024-09-10T18:48:24.715755+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-09-10T18:48:25.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:25 smithi151 bash[22565]: audit 2024-09-10T18:48:24.719639+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:25.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:25 smithi151 bash[22565]: audit 2024-09-10T18:48:24.721308+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[18597]: cluster 2024-09-10T18:48:24.103291+0000 mgr.y (mgr.24512) 548 : cluster [DBG] pgmap v878: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[18597]: audit 2024-09-10T18:48:24.337155+0000 mon.c (mon.2) 484 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[18597]: cluster 2024-09-10T18:48:24.700156+0000 mon.a (mon.0) 2930 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[18597]: cluster 2024-09-10T18:48:24.715755+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[18597]: audit 2024-09-10T18:48:24.719639+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[18597]: audit 2024-09-10T18:48:24.721308+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[31607]: cluster 2024-09-10T18:48:24.103291+0000 mgr.y (mgr.24512) 548 : cluster [DBG] pgmap v878: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[31607]: audit 2024-09-10T18:48:24.337155+0000 mon.c (mon.2) 484 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:25.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[31607]: cluster 2024-09-10T18:48:24.700156+0000 mon.a (mon.0) 2930 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[31607]: cluster 2024-09-10T18:48:24.715755+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-09-10T18:48:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[31607]: audit 2024-09-10T18:48:24.719639+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:25 smithi100 bash[31607]: audit 2024-09-10T18:48:24.721308+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[18597]: audit 2024-09-10T18:48:25.721091+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[18597]: cluster 2024-09-10T18:48:25.721533+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-09-10T18:48:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[18597]: audit 2024-09-10T18:48:25.724374+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[18597]: audit 2024-09-10T18:48:25.725041+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[31607]: audit 2024-09-10T18:48:25.721091+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[31607]: cluster 2024-09-10T18:48:25.721533+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-09-10T18:48:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[31607]: audit 2024-09-10T18:48:25.724374+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:27.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:26 smithi100 bash[31607]: audit 2024-09-10T18:48:25.725041+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:27.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:26 smithi151 bash[22565]: audit 2024-09-10T18:48:25.721091+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:27.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:26 smithi151 bash[22565]: cluster 2024-09-10T18:48:25.721533+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-09-10T18:48:27.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:26 smithi151 bash[22565]: audit 2024-09-10T18:48:25.724374+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:27.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:26 smithi151 bash[22565]: audit 2024-09-10T18:48:25.725041+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[18597]: cluster 2024-09-10T18:48:26.104119+0000 mgr.y (mgr.24512) 549 : cluster [DBG] pgmap v881: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[18597]: audit 2024-09-10T18:48:26.726128+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[18597]: cluster 2024-09-10T18:48:26.726361+0000 mon.a (mon.0) 2937 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-09-10T18:48:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[18597]: audit 2024-09-10T18:48:26.728761+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[18597]: audit 2024-09-10T18:48:26.735505+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:28.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[31607]: cluster 2024-09-10T18:48:26.104119+0000 mgr.y (mgr.24512) 549 : cluster [DBG] pgmap v881: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:28.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[31607]: audit 2024-09-10T18:48:26.726128+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:28.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[31607]: cluster 2024-09-10T18:48:26.726361+0000 mon.a (mon.0) 2937 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-09-10T18:48:28.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[31607]: audit 2024-09-10T18:48:26.728761+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:28.048 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:27 smithi100 bash[31607]: audit 2024-09-10T18:48:26.735505+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:28.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:27 smithi151 bash[22565]: cluster 2024-09-10T18:48:26.104119+0000 mgr.y (mgr.24512) 549 : cluster [DBG] pgmap v881: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:28.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:27 smithi151 bash[22565]: audit 2024-09-10T18:48:26.726128+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:28.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:27 smithi151 bash[22565]: cluster 2024-09-10T18:48:26.726361+0000 mon.a (mon.0) 2937 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-09-10T18:48:28.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:27 smithi151 bash[22565]: audit 2024-09-10T18:48:26.728761+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:28.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:27 smithi151 bash[22565]: audit 2024-09-10T18:48:26.735505+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[18597]: audit 2024-09-10T18:48:27.753254+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[18597]: cluster 2024-09-10T18:48:27.753462+0000 mon.a (mon.0) 2940 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-09-10T18:48:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[18597]: audit 2024-09-10T18:48:27.756466+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:29.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[18597]: audit 2024-09-10T18:48:27.756847+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[31607]: audit 2024-09-10T18:48:27.753254+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[31607]: cluster 2024-09-10T18:48:27.753462+0000 mon.a (mon.0) 2940 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-09-10T18:48:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[31607]: audit 2024-09-10T18:48:27.756466+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:29.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:28 smithi100 bash[31607]: audit 2024-09-10T18:48:27.756847+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:28 smithi151 bash[22565]: audit 2024-09-10T18:48:27.753254+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-109", "overlaypool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:28 smithi151 bash[22565]: cluster 2024-09-10T18:48:27.753462+0000 mon.a (mon.0) 2940 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-09-10T18:48:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:28 smithi151 bash[22565]: audit 2024-09-10T18:48:27.756466+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:28 smithi151 bash[22565]: audit 2024-09-10T18:48:27.756847+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:48:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[18597]: cluster 2024-09-10T18:48:28.105045+0000 mgr.y (mgr.24512) 550 : cluster [DBG] pgmap v884: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[18597]: cluster 2024-09-10T18:48:28.744203+0000 mon.a (mon.0) 2942 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[18597]: audit 2024-09-10T18:48:28.749750+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]': finished 2024-09-10T18:48:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[18597]: cluster 2024-09-10T18:48:28.749853+0000 mon.a (mon.0) 2944 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-09-10T18:48:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[18597]: audit 2024-09-10T18:48:28.752030+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:48:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[18597]: audit 2024-09-10T18:48:28.752717+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:48:30.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[31607]: cluster 2024-09-10T18:48:28.105045+0000 mgr.y (mgr.24512) 550 : cluster [DBG] pgmap v884: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:30.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[31607]: cluster 2024-09-10T18:48:28.744203+0000 mon.a (mon.0) 2942 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:30.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[31607]: audit 2024-09-10T18:48:28.749750+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]': finished 2024-09-10T18:48:30.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[31607]: cluster 2024-09-10T18:48:28.749853+0000 mon.a (mon.0) 2944 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-09-10T18:48:30.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[31607]: audit 2024-09-10T18:48:28.752030+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:48:30.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:29 smithi100 bash[31607]: audit 2024-09-10T18:48:28.752717+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:48:30.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:29 smithi151 bash[22565]: cluster 2024-09-10T18:48:28.105045+0000 mgr.y (mgr.24512) 550 : cluster [DBG] pgmap v884: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:30.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:29 smithi151 bash[22565]: cluster 2024-09-10T18:48:28.744203+0000 mon.a (mon.0) 2942 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:30.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:29 smithi151 bash[22565]: audit 2024-09-10T18:48:28.749750+0000 mon.a (mon.0) 2943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-109-cache", "mode": "writeback"}]': finished 2024-09-10T18:48:30.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:29 smithi151 bash[22565]: cluster 2024-09-10T18:48:28.749853+0000 mon.a (mon.0) 2944 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-09-10T18:48:30.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:29 smithi151 bash[22565]: audit 2024-09-10T18:48:28.752030+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:48:30.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:29 smithi151 bash[22565]: audit 2024-09-10T18:48:28.752717+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:48:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[18597]: audit 2024-09-10T18:48:29.755393+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:48:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[18597]: cluster 2024-09-10T18:48:29.755753+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-09-10T18:48:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[18597]: audit 2024-09-10T18:48:29.756918+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:48:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[18597]: audit 2024-09-10T18:48:29.758387+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:48:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[18597]: audit 2024-09-10T18:48:30.757457+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:48:31.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[18597]: cluster 2024-09-10T18:48:30.757838+0000 mon.a (mon.0) 2950 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-09-10T18:48:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[31607]: audit 2024-09-10T18:48:29.755393+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:48:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[31607]: cluster 2024-09-10T18:48:29.755753+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-09-10T18:48:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[31607]: audit 2024-09-10T18:48:29.756918+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:48:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[31607]: audit 2024-09-10T18:48:29.758387+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:48:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[31607]: audit 2024-09-10T18:48:30.757457+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:48:31.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:30 smithi100 bash[31607]: cluster 2024-09-10T18:48:30.757838+0000 mon.a (mon.0) 2950 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-09-10T18:48:31.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:30 smithi151 bash[22565]: audit 2024-09-10T18:48:29.755393+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:48:31.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:30 smithi151 bash[22565]: cluster 2024-09-10T18:48:29.755753+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-09-10T18:48:31.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:30 smithi151 bash[22565]: audit 2024-09-10T18:48:29.756918+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:48:31.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:30 smithi151 bash[22565]: audit 2024-09-10T18:48:29.758387+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:48:31.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:30 smithi151 bash[22565]: audit 2024-09-10T18:48:30.757457+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:48:31.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:30 smithi151 bash[22565]: cluster 2024-09-10T18:48:30.757838+0000 mon.a (mon.0) 2950 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-09-10T18:48:31.779 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:31] "GET /metrics HTTP/1.1" 200 228644 "" "Prometheus/2.43.0" 2024-09-10T18:48:31.779 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[31607]: cluster 2024-09-10T18:48:30.105954+0000 mgr.y (mgr.24512) 551 : cluster [DBG] pgmap v887: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:31.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[31607]: audit 2024-09-10T18:48:30.761438+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:48:31.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[31607]: audit 2024-09-10T18:48:30.769138+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:48:31.780 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[31607]: cluster 2024-09-10T18:48:31.764534+0000 mon.a (mon.0) 2952 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:32.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[18597]: cluster 2024-09-10T18:48:30.105954+0000 mgr.y (mgr.24512) 551 : cluster [DBG] pgmap v887: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:32.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[18597]: audit 2024-09-10T18:48:30.761438+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:48:32.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[18597]: audit 2024-09-10T18:48:30.769138+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:48:32.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:31 smithi100 bash[18597]: cluster 2024-09-10T18:48:31.764534+0000 mon.a (mon.0) 2952 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:32.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:31 smithi151 bash[22565]: cluster 2024-09-10T18:48:30.105954+0000 mgr.y (mgr.24512) 551 : cluster [DBG] pgmap v887: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:32.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:31 smithi151 bash[22565]: audit 2024-09-10T18:48:30.761438+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:48:32.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:31 smithi151 bash[22565]: audit 2024-09-10T18:48:30.769138+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:48:32.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:31 smithi151 bash[22565]: cluster 2024-09-10T18:48:31.764534+0000 mon.a (mon.0) 2952 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[31607]: audit 2024-09-10T18:48:31.773963+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:48:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[31607]: cluster 2024-09-10T18:48:31.774330+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-09-10T18:48:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[31607]: audit 2024-09-10T18:48:31.775762+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:48:33.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[31607]: audit 2024-09-10T18:48:31.777595+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:48:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[18597]: audit 2024-09-10T18:48:31.773963+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:48:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[18597]: cluster 2024-09-10T18:48:31.774330+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-09-10T18:48:33.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[18597]: audit 2024-09-10T18:48:31.775762+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:48:33.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:32 smithi100 bash[18597]: audit 2024-09-10T18:48:31.777595+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:48:33.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:48:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:48:33.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:32 smithi151 bash[22565]: audit 2024-09-10T18:48:31.773963+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:48:33.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:32 smithi151 bash[22565]: cluster 2024-09-10T18:48:31.774330+0000 mon.a (mon.0) 2954 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-09-10T18:48:33.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:32 smithi151 bash[22565]: audit 2024-09-10T18:48:31.775762+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:48:33.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:32 smithi151 bash[22565]: audit 2024-09-10T18:48:31.777595+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:48:34.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:33 smithi151 bash[22565]: cluster 2024-09-10T18:48:32.106822+0000 mgr.y (mgr.24512) 552 : cluster [DBG] pgmap v890: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:34.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:33 smithi151 bash[22565]: audit 2024-09-10T18:48:32.670822+0000 mgr.y (mgr.24512) 553 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:34.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:33 smithi151 bash[22565]: audit 2024-09-10T18:48:32.781350+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T18:48:34.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:33 smithi151 bash[22565]: cluster 2024-09-10T18:48:32.781827+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-09-10T18:48:34.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:33 smithi151 bash[22565]: audit 2024-09-10T18:48:32.834888+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:34.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:33 smithi151 bash[22565]: audit 2024-09-10T18:48:32.835389+0000 mon.a (mon.0) 2958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[18597]: cluster 2024-09-10T18:48:32.106822+0000 mgr.y (mgr.24512) 552 : cluster [DBG] pgmap v890: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[18597]: audit 2024-09-10T18:48:32.670822+0000 mgr.y (mgr.24512) 553 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[18597]: audit 2024-09-10T18:48:32.781350+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[18597]: cluster 2024-09-10T18:48:32.781827+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[18597]: audit 2024-09-10T18:48:32.834888+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[18597]: audit 2024-09-10T18:48:32.835389+0000 mon.a (mon.0) 2958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[31607]: cluster 2024-09-10T18:48:32.106822+0000 mgr.y (mgr.24512) 552 : cluster [DBG] pgmap v890: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:34.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[31607]: audit 2024-09-10T18:48:32.670822+0000 mgr.y (mgr.24512) 553 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:34.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[31607]: audit 2024-09-10T18:48:32.781350+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T18:48:34.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[31607]: cluster 2024-09-10T18:48:32.781827+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-09-10T18:48:34.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[31607]: audit 2024-09-10T18:48:32.834888+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:34.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:33 smithi100 bash[31607]: audit 2024-09-10T18:48:32.835389+0000 mon.a (mon.0) 2958 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:34.828 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:48:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:35.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:34 smithi151 bash[22565]: audit 2024-09-10T18:48:33.812168+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:35.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:34 smithi151 bash[22565]: cluster 2024-09-10T18:48:33.812290+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-09-10T18:48:35.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:34 smithi151 bash[22565]: audit 2024-09-10T18:48:33.813272+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:35.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:34 smithi151 bash[22565]: audit 2024-09-10T18:48:33.814720+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[18597]: audit 2024-09-10T18:48:33.812168+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[18597]: cluster 2024-09-10T18:48:33.812290+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-09-10T18:48:35.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[18597]: audit 2024-09-10T18:48:33.813272+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:35.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[18597]: audit 2024-09-10T18:48:33.814720+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:35.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[31607]: audit 2024-09-10T18:48:33.812168+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:35.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[31607]: cluster 2024-09-10T18:48:33.812290+0000 mon.a (mon.0) 2960 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-09-10T18:48:35.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[31607]: audit 2024-09-10T18:48:33.813272+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:35.296 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:34 smithi100 bash[31607]: audit 2024-09-10T18:48:33.814720+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]: dispatch 2024-09-10T18:48:36.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:35 smithi151 bash[22565]: cluster 2024-09-10T18:48:34.107696+0000 mgr.y (mgr.24512) 554 : cluster [DBG] pgmap v893: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:36.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:35 smithi151 bash[22565]: audit 2024-09-10T18:48:34.821299+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:36.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:35 smithi151 bash[22565]: cluster 2024-09-10T18:48:34.821542+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-09-10T18:48:36.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:35 smithi100 bash[18597]: cluster 2024-09-10T18:48:34.107696+0000 mgr.y (mgr.24512) 554 : cluster [DBG] pgmap v893: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:36.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:35 smithi100 bash[18597]: audit 2024-09-10T18:48:34.821299+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:36.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:35 smithi100 bash[18597]: cluster 2024-09-10T18:48:34.821542+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-09-10T18:48:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:35 smithi100 bash[31607]: cluster 2024-09-10T18:48:34.107696+0000 mgr.y (mgr.24512) 554 : cluster [DBG] pgmap v893: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:35 smithi100 bash[31607]: audit 2024-09-10T18:48:34.821299+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-109", "tierpool": "test-rados-api-smithi100-72764-109-cache"}]': finished 2024-09-10T18:48:36.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:35 smithi100 bash[31607]: cluster 2024-09-10T18:48:34.821542+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-09-10T18:48:37.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:36 smithi151 bash[22565]: cluster 2024-09-10T18:48:35.835833+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-09-10T18:48:37.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:36 smithi151 bash[22565]: cluster 2024-09-10T18:48:36.836473+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-09-10T18:48:37.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:36 smithi151 bash[22565]: audit 2024-09-10T18:48:36.842891+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:37.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:36 smithi151 bash[22565]: audit 2024-09-10T18:48:36.843302+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[18597]: cluster 2024-09-10T18:48:35.835833+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[18597]: cluster 2024-09-10T18:48:36.836473+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[18597]: audit 2024-09-10T18:48:36.842891+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[18597]: audit 2024-09-10T18:48:36.843302+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[31607]: cluster 2024-09-10T18:48:35.835833+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[31607]: cluster 2024-09-10T18:48:36.836473+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[31607]: audit 2024-09-10T18:48:36.842891+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:37.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:36 smithi100 bash[31607]: audit 2024-09-10T18:48:36.843302+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:38.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:37 smithi151 bash[22565]: cluster 2024-09-10T18:48:36.108526+0000 mgr.y (mgr.24512) 555 : cluster [DBG] pgmap v896: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:38.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:37 smithi151 bash[22565]: audit 2024-09-10T18:48:37.842401+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:37 smithi151 bash[22565]: cluster 2024-09-10T18:48:37.842524+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-09-10T18:48:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:37 smithi151 bash[22565]: audit 2024-09-10T18:48:37.845614+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:37 smithi151 bash[22565]: audit 2024-09-10T18:48:37.846201+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[18597]: cluster 2024-09-10T18:48:36.108526+0000 mgr.y (mgr.24512) 555 : cluster [DBG] pgmap v896: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[18597]: audit 2024-09-10T18:48:37.842401+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[18597]: cluster 2024-09-10T18:48:37.842524+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-09-10T18:48:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[18597]: audit 2024-09-10T18:48:37.845614+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:38.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[18597]: audit 2024-09-10T18:48:37.846201+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[31607]: cluster 2024-09-10T18:48:36.108526+0000 mgr.y (mgr.24512) 555 : cluster [DBG] pgmap v896: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[31607]: audit 2024-09-10T18:48:37.842401+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[31607]: cluster 2024-09-10T18:48:37.842524+0000 mon.a (mon.0) 2968 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-09-10T18:48:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[31607]: audit 2024-09-10T18:48:37.845614+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.100:0/2858941021' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:38.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:37 smithi100 bash[31607]: audit 2024-09-10T18:48:37.846201+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]: dispatch 2024-09-10T18:48:40.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:39 smithi151 bash[22565]: cluster 2024-09-10T18:48:38.109307+0000 mgr.y (mgr.24512) 556 : cluster [DBG] pgmap v899: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:39 smithi151 bash[22565]: audit 2024-09-10T18:48:38.847773+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:39 smithi151 bash[22565]: cluster 2024-09-10T18:48:38.848249+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-09-10T18:48:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:39 smithi151 bash[22565]: cluster 2024-09-10T18:48:38.851887+0000 mon.a (mon.0) 2972 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:39 smithi151 bash[22565]: audit 2024-09-10T18:48:39.337663+0000 mon.c (mon.2) 497 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[18597]: cluster 2024-09-10T18:48:38.109307+0000 mgr.y (mgr.24512) 556 : cluster [DBG] pgmap v899: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[18597]: audit 2024-09-10T18:48:38.847773+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[18597]: cluster 2024-09-10T18:48:38.848249+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[18597]: cluster 2024-09-10T18:48:38.851887+0000 mon.a (mon.0) 2972 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[18597]: audit 2024-09-10T18:48:39.337663+0000 mon.c (mon.2) 497 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[31607]: cluster 2024-09-10T18:48:38.109307+0000 mgr.y (mgr.24512) 556 : cluster [DBG] pgmap v899: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[31607]: audit 2024-09-10T18:48:38.847773+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-109"}]': finished 2024-09-10T18:48:40.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[31607]: cluster 2024-09-10T18:48:38.848249+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-09-10T18:48:40.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[31607]: cluster 2024-09-10T18:48:38.851887+0000 mon.a (mon.0) 2972 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:40.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:39 smithi100 bash[31607]: audit 2024-09-10T18:48:39.337663+0000 mon.c (mon.2) 497 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:41.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:40 smithi151 bash[22565]: cluster 2024-09-10T18:48:39.859524+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-09-10T18:48:41.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:40 smithi151 bash[22565]: audit 2024-09-10T18:48:39.865325+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:41.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:40 smithi151 bash[22565]: audit 2024-09-10T18:48:39.866214+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:40 smithi100 bash[18597]: cluster 2024-09-10T18:48:39.859524+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-09-10T18:48:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:40 smithi100 bash[18597]: audit 2024-09-10T18:48:39.865325+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:41.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:40 smithi100 bash[18597]: audit 2024-09-10T18:48:39.866214+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:40 smithi100 bash[31607]: cluster 2024-09-10T18:48:39.859524+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-09-10T18:48:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:40 smithi100 bash[31607]: audit 2024-09-10T18:48:39.865325+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:41.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:40 smithi100 bash[31607]: audit 2024-09-10T18:48:39.866214+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:41] "GET /metrics HTTP/1.1" 200 226244 "" "Prometheus/2.43.0" 2024-09-10T18:48:42.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:41 smithi151 bash[22565]: cluster 2024-09-10T18:48:40.110123+0000 mgr.y (mgr.24512) 557 : cluster [DBG] pgmap v902: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:42.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:41 smithi151 bash[22565]: audit 2024-09-10T18:48:40.867822+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:42.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:41 smithi151 bash[22565]: cluster 2024-09-10T18:48:40.867938+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-09-10T18:48:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:41 smithi151 bash[22565]: audit 2024-09-10T18:48:40.868869+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:41 smithi151 bash[22565]: audit 2024-09-10T18:48:40.869349+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:41 smithi151 bash[22565]: audit 2024-09-10T18:48:41.868497+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:41 smithi151 bash[22565]: cluster 2024-09-10T18:48:41.868756+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[31607]: cluster 2024-09-10T18:48:40.110123+0000 mgr.y (mgr.24512) 557 : cluster [DBG] pgmap v902: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[31607]: audit 2024-09-10T18:48:40.867822+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[31607]: cluster 2024-09-10T18:48:40.867938+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[31607]: audit 2024-09-10T18:48:40.868869+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[31607]: audit 2024-09-10T18:48:40.869349+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[31607]: audit 2024-09-10T18:48:41.868497+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[31607]: cluster 2024-09-10T18:48:41.868756+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-09-10T18:48:42.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18597]: cluster 2024-09-10T18:48:40.110123+0000 mgr.y (mgr.24512) 557 : cluster [DBG] pgmap v902: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:42.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18597]: audit 2024-09-10T18:48:40.867822+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:42.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18597]: cluster 2024-09-10T18:48:40.867938+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-09-10T18:48:42.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18597]: audit 2024-09-10T18:48:40.868869+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.100:0/916704913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:42.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18597]: audit 2024-09-10T18:48:40.869349+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]: dispatch 2024-09-10T18:48:42.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18597]: audit 2024-09-10T18:48:41.868497+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-72764-104"}]': finished 2024-09-10T18:48:42.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:41 smithi100 bash[18597]: cluster 2024-09-10T18:48:41.868756+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-09-10T18:48:43.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:48:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:48:43.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:42 smithi151 bash[22565]: audit 2024-09-10T18:48:41.891593+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:43.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:42 smithi151 bash[22565]: audit 2024-09-10T18:48:41.894039+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:43.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:42 smithi151 bash[22565]: audit 2024-09-10T18:48:41.894575+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:43.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:42 smithi100 bash[18597]: audit 2024-09-10T18:48:41.891593+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:43.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:42 smithi100 bash[18597]: audit 2024-09-10T18:48:41.894039+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:43.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:42 smithi100 bash[18597]: audit 2024-09-10T18:48:41.894575+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:43.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:42 smithi100 bash[31607]: audit 2024-09-10T18:48:41.891593+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:43.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:42 smithi100 bash[31607]: audit 2024-09-10T18:48:41.894039+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:43.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:42 smithi100 bash[31607]: audit 2024-09-10T18:48:41.894575+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:48:44.671 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:48:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:44.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:44 smithi151 bash[22565]: cluster 2024-09-10T18:48:42.110999+0000 mgr.y (mgr.24512) 558 : cluster [DBG] pgmap v905: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:48:44.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:44 smithi151 bash[22565]: audit 2024-09-10T18:48:42.682169+0000 mgr.y (mgr.24512) 559 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:44.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:44 smithi151 bash[22565]: audit 2024-09-10T18:48:42.889549+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:44.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:44 smithi151 bash[22565]: cluster 2024-09-10T18:48:42.889733+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-09-10T18:48:44.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:44 smithi151 bash[22565]: audit 2024-09-10T18:48:42.890611+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[18597]: cluster 2024-09-10T18:48:42.110999+0000 mgr.y (mgr.24512) 558 : cluster [DBG] pgmap v905: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:48:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[18597]: audit 2024-09-10T18:48:42.682169+0000 mgr.y (mgr.24512) 559 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[18597]: audit 2024-09-10T18:48:42.889549+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[18597]: cluster 2024-09-10T18:48:42.889733+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-09-10T18:48:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[18597]: audit 2024-09-10T18:48:42.890611+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[31607]: cluster 2024-09-10T18:48:42.110999+0000 mgr.y (mgr.24512) 558 : cluster [DBG] pgmap v905: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:48:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[31607]: audit 2024-09-10T18:48:42.682169+0000 mgr.y (mgr.24512) 559 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[31607]: audit 2024-09-10T18:48:42.889549+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-72764-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:48:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[31607]: cluster 2024-09-10T18:48:42.889733+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-09-10T18:48:44.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:44 smithi100 bash[31607]: audit 2024-09-10T18:48:42.890611+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:45.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:45 smithi151 bash[22565]: cluster 2024-09-10T18:48:44.316842+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-09-10T18:48:45.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:45 smithi151 bash[22565]: audit 2024-09-10T18:48:44.901833+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:45.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:45 smithi151 bash[22565]: cluster 2024-09-10T18:48:44.901939+0000 mon.a (mon.0) 2988 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-09-10T18:48:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:45 smithi100 bash[18597]: cluster 2024-09-10T18:48:44.316842+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-09-10T18:48:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:45 smithi100 bash[18597]: audit 2024-09-10T18:48:44.901833+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:45 smithi100 bash[18597]: cluster 2024-09-10T18:48:44.901939+0000 mon.a (mon.0) 2988 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-09-10T18:48:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:45 smithi100 bash[31607]: cluster 2024-09-10T18:48:44.316842+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-09-10T18:48:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:45 smithi100 bash[31607]: audit 2024-09-10T18:48:44.901833+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-72764-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:45 smithi100 bash[31607]: cluster 2024-09-10T18:48:44.901939+0000 mon.a (mon.0) 2988 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-09-10T18:48:46.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:46 smithi151 bash[22565]: cluster 2024-09-10T18:48:44.111729+0000 mgr.y (mgr.24512) 560 : cluster [DBG] pgmap v908: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:46.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:46 smithi151 bash[22565]: cluster 2024-09-10T18:48:45.906363+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-09-10T18:48:46.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:46 smithi151 bash[22565]: audit 2024-09-10T18:48:45.918860+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:46 smithi151 bash[22565]: cluster 2024-09-10T18:48:46.086237+0000 mon.a (mon.0) 2991 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[18597]: cluster 2024-09-10T18:48:44.111729+0000 mgr.y (mgr.24512) 560 : cluster [DBG] pgmap v908: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[18597]: cluster 2024-09-10T18:48:45.906363+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-09-10T18:48:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[18597]: audit 2024-09-10T18:48:45.918860+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[18597]: cluster 2024-09-10T18:48:46.086237+0000 mon.a (mon.0) 2991 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[31607]: cluster 2024-09-10T18:48:44.111729+0000 mgr.y (mgr.24512) 560 : cluster [DBG] pgmap v908: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[31607]: cluster 2024-09-10T18:48:45.906363+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-09-10T18:48:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[31607]: audit 2024-09-10T18:48:45.918860+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:46 smithi100 bash[31607]: cluster 2024-09-10T18:48:46.086237+0000 mon.a (mon.0) 2991 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:47.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: cluster 2024-09-10T18:48:46.112566+0000 mgr.y (mgr.24512) 561 : cluster [DBG] pgmap v911: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:47.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: audit 2024-09-10T18:48:46.792143+0000 mon.c (mon.2) 500 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:47.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: audit 2024-09-10T18:48:46.908147+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:47.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: cluster 2024-09-10T18:48:46.908367+0000 mon.a (mon.0) 2993 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-09-10T18:48:47.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: audit 2024-09-10T18:48:46.937476+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:48:47.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: audit 2024-09-10T18:48:47.256689+0000 mon.c (mon.2) 501 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:47.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: audit 2024-09-10T18:48:47.258015+0000 mon.c (mon.2) 502 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:47.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:47 smithi151 bash[22565]: audit 2024-09-10T18:48:47.266797+0000 mon.a (mon.0) 2995 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:48:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: cluster 2024-09-10T18:48:46.112566+0000 mgr.y (mgr.24512) 561 : cluster [DBG] pgmap v911: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: audit 2024-09-10T18:48:46.792143+0000 mon.c (mon.2) 500 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: audit 2024-09-10T18:48:46.908147+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: cluster 2024-09-10T18:48:46.908367+0000 mon.a (mon.0) 2993 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-09-10T18:48:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: audit 2024-09-10T18:48:46.937476+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:48:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: audit 2024-09-10T18:48:47.256689+0000 mon.c (mon.2) 501 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: audit 2024-09-10T18:48:47.258015+0000 mon.c (mon.2) 502 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:47.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[18597]: audit 2024-09-10T18:48:47.266797+0000 mon.a (mon.0) 2995 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:48:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: cluster 2024-09-10T18:48:46.112566+0000 mgr.y (mgr.24512) 561 : cluster [DBG] pgmap v911: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: audit 2024-09-10T18:48:46.792143+0000 mon.c (mon.2) 500 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: audit 2024-09-10T18:48:46.908147+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: cluster 2024-09-10T18:48:46.908367+0000 mon.a (mon.0) 2993 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-09-10T18:48:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: audit 2024-09-10T18:48:46.937476+0000 mon.a (mon.0) 2994 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:48:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: audit 2024-09-10T18:48:47.256689+0000 mon.c (mon.2) 501 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:47.796 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: audit 2024-09-10T18:48:47.258015+0000 mon.c (mon.2) 502 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:47.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:47 smithi100 bash[31607]: audit 2024-09-10T18:48:47.266797+0000 mon.a (mon.0) 2995 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:48:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:48 smithi100 bash[18597]: audit 2024-09-10T18:48:47.925435+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:48:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:48 smithi100 bash[18597]: cluster 2024-09-10T18:48:47.925594+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-09-10T18:48:49.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:48 smithi100 bash[18597]: audit 2024-09-10T18:48:47.926580+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-112"}]: dispatch 2024-09-10T18:48:49.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:48 smithi100 bash[31607]: audit 2024-09-10T18:48:47.925435+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:48:49.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:48 smithi100 bash[31607]: cluster 2024-09-10T18:48:47.925594+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-09-10T18:48:49.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:48 smithi100 bash[31607]: audit 2024-09-10T18:48:47.926580+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-112"}]: dispatch 2024-09-10T18:48:49.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:48 smithi151 bash[22565]: audit 2024-09-10T18:48:47.925435+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:48:49.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:48 smithi151 bash[22565]: cluster 2024-09-10T18:48:47.925594+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-09-10T18:48:49.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:48 smithi151 bash[22565]: audit 2024-09-10T18:48:47.926580+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-112"}]: dispatch 2024-09-10T18:48:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[18597]: cluster 2024-09-10T18:48:48.113478+0000 mgr.y (mgr.24512) 562 : cluster [DBG] pgmap v914: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[18597]: audit 2024-09-10T18:48:48.935867+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-112"}]': finished 2024-09-10T18:48:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[18597]: cluster 2024-09-10T18:48:48.935969+0000 mon.a (mon.0) 3000 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-09-10T18:48:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[18597]: audit 2024-09-10T18:48:48.974144+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[31607]: cluster 2024-09-10T18:48:48.113478+0000 mgr.y (mgr.24512) 562 : cluster [DBG] pgmap v914: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[31607]: audit 2024-09-10T18:48:48.935867+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-112"}]': finished 2024-09-10T18:48:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[31607]: cluster 2024-09-10T18:48:48.935969+0000 mon.a (mon.0) 3000 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-09-10T18:48:50.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:49 smithi100 bash[31607]: audit 2024-09-10T18:48:48.974144+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:50.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:49 smithi151 bash[22565]: cluster 2024-09-10T18:48:48.113478+0000 mgr.y (mgr.24512) 562 : cluster [DBG] pgmap v914: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:50.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:49 smithi151 bash[22565]: audit 2024-09-10T18:48:48.935867+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-112"}]': finished 2024-09-10T18:48:50.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:49 smithi151 bash[22565]: cluster 2024-09-10T18:48:48.935969+0000 mon.a (mon.0) 3000 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-09-10T18:48:50.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:49 smithi151 bash[22565]: audit 2024-09-10T18:48:48.974144+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:51.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[18597]: audit 2024-09-10T18:48:49.937769+0000 mon.a (mon.0) 3002 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:51.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[18597]: cluster 2024-09-10T18:48:49.937988+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-09-10T18:48:51.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[18597]: audit 2024-09-10T18:48:49.939671+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112"}]: dispatch 2024-09-10T18:48:51.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[18597]: audit 2024-09-10T18:48:50.940303+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112"}]': finished 2024-09-10T18:48:51.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[31607]: audit 2024-09-10T18:48:49.937769+0000 mon.a (mon.0) 3002 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:51.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[31607]: cluster 2024-09-10T18:48:49.937988+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-09-10T18:48:51.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[31607]: audit 2024-09-10T18:48:49.939671+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112"}]: dispatch 2024-09-10T18:48:51.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:50 smithi100 bash[31607]: audit 2024-09-10T18:48:50.940303+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112"}]': finished 2024-09-10T18:48:51.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:50 smithi151 bash[22565]: audit 2024-09-10T18:48:49.937769+0000 mon.a (mon.0) 3002 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:51.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:50 smithi151 bash[22565]: cluster 2024-09-10T18:48:49.937988+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-09-10T18:48:51.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:50 smithi151 bash[22565]: audit 2024-09-10T18:48:49.939671+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112"}]: dispatch 2024-09-10T18:48:51.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:50 smithi151 bash[22565]: audit 2024-09-10T18:48:50.940303+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-112"}]': finished 2024-09-10T18:48:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:48:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:51] "GET /metrics HTTP/1.1" 200 226244 "" "Prometheus/2.43.0" 2024-09-10T18:48:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:51 smithi100 bash[18597]: cluster 2024-09-10T18:48:50.114234+0000 mgr.y (mgr.24512) 563 : cluster [DBG] pgmap v917: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:51 smithi100 bash[18597]: cluster 2024-09-10T18:48:50.940473+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-09-10T18:48:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:51 smithi100 bash[18597]: cluster 2024-09-10T18:48:51.087560+0000 mon.a (mon.0) 3007 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:51 smithi100 bash[31607]: cluster 2024-09-10T18:48:50.114234+0000 mgr.y (mgr.24512) 563 : cluster [DBG] pgmap v917: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:51 smithi100 bash[31607]: cluster 2024-09-10T18:48:50.940473+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-09-10T18:48:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:51 smithi100 bash[31607]: cluster 2024-09-10T18:48:51.087560+0000 mon.a (mon.0) 3007 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:52.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:51 smithi151 bash[22565]: cluster 2024-09-10T18:48:50.114234+0000 mgr.y (mgr.24512) 563 : cluster [DBG] pgmap v917: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:52.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:51 smithi151 bash[22565]: cluster 2024-09-10T18:48:50.940473+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-09-10T18:48:52.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:51 smithi151 bash[22565]: cluster 2024-09-10T18:48:51.087560+0000 mon.a (mon.0) 3007 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:52.968 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:48:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:48:53.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:52 smithi100 bash[18597]: cluster 2024-09-10T18:48:51.961485+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-09-10T18:48:53.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:52 smithi100 bash[31607]: cluster 2024-09-10T18:48:51.961485+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-09-10T18:48:53.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:52 smithi151 bash[22565]: cluster 2024-09-10T18:48:51.961485+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[18597]: cluster 2024-09-10T18:48:52.114980+0000 mgr.y (mgr.24512) 564 : cluster [DBG] pgmap v920: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[18597]: audit 2024-09-10T18:48:52.693606+0000 mgr.y (mgr.24512) 565 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[18597]: cluster 2024-09-10T18:48:52.971607+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[18597]: audit 2024-09-10T18:48:52.973262+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[31607]: cluster 2024-09-10T18:48:52.114980+0000 mgr.y (mgr.24512) 564 : cluster [DBG] pgmap v920: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[31607]: audit 2024-09-10T18:48:52.693606+0000 mgr.y (mgr.24512) 565 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[31607]: cluster 2024-09-10T18:48:52.971607+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-09-10T18:48:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:53 smithi100 bash[31607]: audit 2024-09-10T18:48:52.973262+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:54.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:53 smithi151 bash[22565]: cluster 2024-09-10T18:48:52.114980+0000 mgr.y (mgr.24512) 564 : cluster [DBG] pgmap v920: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:54.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:53 smithi151 bash[22565]: audit 2024-09-10T18:48:52.693606+0000 mgr.y (mgr.24512) 565 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:54.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:53 smithi151 bash[22565]: cluster 2024-09-10T18:48:52.971607+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-09-10T18:48:54.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:53 smithi151 bash[22565]: audit 2024-09-10T18:48:52.973262+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:54.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:48:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:48:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:54 smithi100 bash[18597]: audit 2024-09-10T18:48:53.977416+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:54 smithi100 bash[18597]: cluster 2024-09-10T18:48:53.977960+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:54 smithi100 bash[18597]: audit 2024-09-10T18:48:53.998636+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:54 smithi100 bash[18597]: audit 2024-09-10T18:48:54.338096+0000 mon.c (mon.2) 503 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:55 smithi100 bash[31607]: audit 2024-09-10T18:48:53.977416+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:55 smithi100 bash[31607]: cluster 2024-09-10T18:48:53.977960+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:55 smithi100 bash[31607]: audit 2024-09-10T18:48:53.998636+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:48:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:55 smithi100 bash[31607]: audit 2024-09-10T18:48:54.338096+0000 mon.c (mon.2) 503 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:55.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:54 smithi151 bash[22565]: audit 2024-09-10T18:48:53.977416+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:55.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:54 smithi151 bash[22565]: cluster 2024-09-10T18:48:53.977960+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-09-10T18:48:55.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:54 smithi151 bash[22565]: audit 2024-09-10T18:48:53.998636+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:48:55.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:54 smithi151 bash[22565]: audit 2024-09-10T18:48:54.338096+0000 mon.c (mon.2) 503 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[18597]: cluster 2024-09-10T18:48:54.115784+0000 mgr.y (mgr.24512) 566 : cluster [DBG] pgmap v923: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[18597]: audit 2024-09-10T18:48:55.005468+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[18597]: cluster 2024-09-10T18:48:55.005541+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[18597]: audit 2024-09-10T18:48:55.009016+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-114"}]: dispatch 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[31607]: cluster 2024-09-10T18:48:54.115784+0000 mgr.y (mgr.24512) 566 : cluster [DBG] pgmap v923: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[31607]: audit 2024-09-10T18:48:55.005468+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[31607]: cluster 2024-09-10T18:48:55.005541+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-09-10T18:48:56.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:56 smithi100 bash[31607]: audit 2024-09-10T18:48:55.009016+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-114"}]: dispatch 2024-09-10T18:48:56.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:55 smithi151 bash[22565]: cluster 2024-09-10T18:48:54.115784+0000 mgr.y (mgr.24512) 566 : cluster [DBG] pgmap v923: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:56.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:55 smithi151 bash[22565]: audit 2024-09-10T18:48:55.005468+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:48:56.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:55 smithi151 bash[22565]: cluster 2024-09-10T18:48:55.005541+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-09-10T18:48:56.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:55 smithi151 bash[22565]: audit 2024-09-10T18:48:55.009016+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-114"}]: dispatch 2024-09-10T18:48:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:57 smithi100 bash[31607]: audit 2024-09-10T18:48:56.015907+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-114"}]': finished 2024-09-10T18:48:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:57 smithi100 bash[31607]: cluster 2024-09-10T18:48:56.016046+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-09-10T18:48:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:57 smithi100 bash[31607]: audit 2024-09-10T18:48:56.017720+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-114", "mode": "writeback"}]: dispatch 2024-09-10T18:48:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:57 smithi100 bash[18597]: audit 2024-09-10T18:48:56.015907+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-114"}]': finished 2024-09-10T18:48:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:57 smithi100 bash[18597]: cluster 2024-09-10T18:48:56.016046+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-09-10T18:48:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:57 smithi100 bash[18597]: audit 2024-09-10T18:48:56.017720+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-114", "mode": "writeback"}]: dispatch 2024-09-10T18:48:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:57 smithi151 bash[22565]: audit 2024-09-10T18:48:56.015907+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-114"}]': finished 2024-09-10T18:48:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:57 smithi151 bash[22565]: cluster 2024-09-10T18:48:56.016046+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-09-10T18:48:57.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:57 smithi151 bash[22565]: audit 2024-09-10T18:48:56.017720+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-114", "mode": "writeback"}]: dispatch 2024-09-10T18:48:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[18597]: cluster 2024-09-10T18:48:56.116830+0000 mgr.y (mgr.24512) 567 : cluster [DBG] pgmap v926: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[18597]: cluster 2024-09-10T18:48:57.011261+0000 mon.a (mon.0) 3020 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[18597]: audit 2024-09-10T18:48:57.027880+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-114", "mode": "writeback"}]': finished 2024-09-10T18:48:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[18597]: cluster 2024-09-10T18:48:57.027995+0000 mon.a (mon.0) 3022 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-09-10T18:48:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[18597]: audit 2024-09-10T18:48:57.083031+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[31607]: cluster 2024-09-10T18:48:56.116830+0000 mgr.y (mgr.24512) 567 : cluster [DBG] pgmap v926: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:58.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[31607]: cluster 2024-09-10T18:48:57.011261+0000 mon.a (mon.0) 3020 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:58.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[31607]: audit 2024-09-10T18:48:57.027880+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-114", "mode": "writeback"}]': finished 2024-09-10T18:48:58.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[31607]: cluster 2024-09-10T18:48:57.027995+0000 mon.a (mon.0) 3022 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-09-10T18:48:58.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:58 smithi100 bash[31607]: audit 2024-09-10T18:48:57.083031+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:58.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:58 smithi151 bash[22565]: cluster 2024-09-10T18:48:56.116830+0000 mgr.y (mgr.24512) 567 : cluster [DBG] pgmap v926: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:48:58.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:58 smithi151 bash[22565]: cluster 2024-09-10T18:48:57.011261+0000 mon.a (mon.0) 3020 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:48:58.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:58 smithi151 bash[22565]: audit 2024-09-10T18:48:57.027880+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-114", "mode": "writeback"}]': finished 2024-09-10T18:48:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:58 smithi151 bash[22565]: cluster 2024-09-10T18:48:57.027995+0000 mon.a (mon.0) 3022 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-09-10T18:48:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:58 smithi151 bash[22565]: audit 2024-09-10T18:48:57.083031+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:48:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[18597]: audit 2024-09-10T18:48:58.029761+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[18597]: cluster 2024-09-10T18:48:58.029923+0000 mon.a (mon.0) 3025 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-09-10T18:48:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[18597]: audit 2024-09-10T18:48:58.033131+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114"}]: dispatch 2024-09-10T18:48:59.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[18597]: cluster 2024-09-10T18:48:59.022704+0000 mon.a (mon.0) 3027 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[31607]: audit 2024-09-10T18:48:58.029761+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[31607]: cluster 2024-09-10T18:48:58.029923+0000 mon.a (mon.0) 3025 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-09-10T18:48:59.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[31607]: audit 2024-09-10T18:48:58.033131+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114"}]: dispatch 2024-09-10T18:48:59.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:48:59 smithi100 bash[31607]: cluster 2024-09-10T18:48:59.022704+0000 mon.a (mon.0) 3027 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:48:59.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:59 smithi151 bash[22565]: audit 2024-09-10T18:48:58.029761+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:48:59.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:59 smithi151 bash[22565]: cluster 2024-09-10T18:48:58.029923+0000 mon.a (mon.0) 3025 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-09-10T18:48:59.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:59 smithi151 bash[22565]: audit 2024-09-10T18:48:58.033131+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114"}]: dispatch 2024-09-10T18:48:59.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:48:59 smithi151 bash[22565]: cluster 2024-09-10T18:48:59.022704+0000 mon.a (mon.0) 3027 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:00.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:00 smithi151 bash[22565]: cluster 2024-09-10T18:48:58.117750+0000 mgr.y (mgr.24512) 568 : cluster [DBG] pgmap v929: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:00.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:00 smithi151 bash[22565]: audit 2024-09-10T18:48:59.034906+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114"}]': finished 2024-09-10T18:49:00.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:00 smithi151 bash[22565]: cluster 2024-09-10T18:48:59.034989+0000 mon.a (mon.0) 3029 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-09-10T18:49:00.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:00 smithi100 bash[18597]: cluster 2024-09-10T18:48:58.117750+0000 mgr.y (mgr.24512) 568 : cluster [DBG] pgmap v929: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:00.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:00 smithi100 bash[18597]: audit 2024-09-10T18:48:59.034906+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114"}]': finished 2024-09-10T18:49:00.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:00 smithi100 bash[18597]: cluster 2024-09-10T18:48:59.034989+0000 mon.a (mon.0) 3029 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-09-10T18:49:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:00 smithi100 bash[31607]: cluster 2024-09-10T18:48:58.117750+0000 mgr.y (mgr.24512) 568 : cluster [DBG] pgmap v929: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:00 smithi100 bash[31607]: audit 2024-09-10T18:48:59.034906+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-114"}]': finished 2024-09-10T18:49:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:00 smithi100 bash[31607]: cluster 2024-09-10T18:48:59.034989+0000 mon.a (mon.0) 3029 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-09-10T18:49:01.341 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:01 smithi100 bash[18597]: cluster 2024-09-10T18:49:00.064262+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-09-10T18:49:01.341 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:01 smithi100 bash[31607]: cluster 2024-09-10T18:49:00.064262+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-09-10T18:49:01.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:01 smithi151 bash[22565]: cluster 2024-09-10T18:49:00.064262+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-09-10T18:49:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:49:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:01] "GET /metrics HTTP/1.1" 200 228689 "" "Prometheus/2.43.0" 2024-09-10T18:49:02.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:02 smithi151 bash[22565]: cluster 2024-09-10T18:49:00.118407+0000 mgr.y (mgr.24512) 569 : cluster [DBG] pgmap v932: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:02.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:02 smithi151 bash[22565]: cluster 2024-09-10T18:49:01.077383+0000 mon.a (mon.0) 3031 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-09-10T18:49:02.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:02 smithi151 bash[22565]: audit 2024-09-10T18:49:01.082091+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:02 smithi100 bash[18597]: cluster 2024-09-10T18:49:00.118407+0000 mgr.y (mgr.24512) 569 : cluster [DBG] pgmap v932: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:02 smithi100 bash[18597]: cluster 2024-09-10T18:49:01.077383+0000 mon.a (mon.0) 3031 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-09-10T18:49:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:02 smithi100 bash[18597]: audit 2024-09-10T18:49:01.082091+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:02 smithi100 bash[31607]: cluster 2024-09-10T18:49:00.118407+0000 mgr.y (mgr.24512) 569 : cluster [DBG] pgmap v932: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:02 smithi100 bash[31607]: cluster 2024-09-10T18:49:01.077383+0000 mon.a (mon.0) 3031 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-09-10T18:49:02.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:02 smithi100 bash[31607]: audit 2024-09-10T18:49:01.082091+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:03.095 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:49:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:49:03.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:03 smithi151 bash[22565]: audit 2024-09-10T18:49:02.079207+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:03.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:03 smithi151 bash[22565]: cluster 2024-09-10T18:49:02.079877+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-09-10T18:49:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:03 smithi100 bash[18597]: audit 2024-09-10T18:49:02.079207+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:03 smithi100 bash[18597]: cluster 2024-09-10T18:49:02.079877+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-09-10T18:49:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:03 smithi100 bash[31607]: audit 2024-09-10T18:49:02.079207+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:03 smithi100 bash[31607]: cluster 2024-09-10T18:49:02.079877+0000 mon.a (mon.0) 3034 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-09-10T18:49:04.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:04 smithi151 bash[22565]: cluster 2024-09-10T18:49:02.118974+0000 mgr.y (mgr.24512) 570 : cluster [DBG] pgmap v935: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:04.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:04 smithi151 bash[22565]: audit 2024-09-10T18:49:02.704992+0000 mgr.y (mgr.24512) 571 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:04.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:04 smithi151 bash[22565]: cluster 2024-09-10T18:49:03.102571+0000 mon.a (mon.0) 3035 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-09-10T18:49:04.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:04 smithi151 bash[22565]: audit 2024-09-10T18:49:03.125348+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[18597]: cluster 2024-09-10T18:49:02.118974+0000 mgr.y (mgr.24512) 570 : cluster [DBG] pgmap v935: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[18597]: audit 2024-09-10T18:49:02.704992+0000 mgr.y (mgr.24512) 571 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[18597]: cluster 2024-09-10T18:49:03.102571+0000 mon.a (mon.0) 3035 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[18597]: audit 2024-09-10T18:49:03.125348+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[31607]: cluster 2024-09-10T18:49:02.118974+0000 mgr.y (mgr.24512) 570 : cluster [DBG] pgmap v935: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[31607]: audit 2024-09-10T18:49:02.704992+0000 mgr.y (mgr.24512) 571 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[31607]: cluster 2024-09-10T18:49:03.102571+0000 mon.a (mon.0) 3035 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-09-10T18:49:04.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:04 smithi100 bash[31607]: audit 2024-09-10T18:49:03.125348+0000 mon.a (mon.0) 3036 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:04.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:49:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:05.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:05 smithi151 bash[22565]: audit 2024-09-10T18:49:04.103600+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:05.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:05 smithi151 bash[22565]: cluster 2024-09-10T18:49:04.103779+0000 mon.a (mon.0) 3038 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-09-10T18:49:05.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:05 smithi151 bash[22565]: audit 2024-09-10T18:49:04.104690+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-116"}]: dispatch 2024-09-10T18:49:05.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:05 smithi151 bash[22565]: audit 2024-09-10T18:49:05.106894+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-116"}]': finished 2024-09-10T18:49:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[18597]: audit 2024-09-10T18:49:04.103600+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[18597]: cluster 2024-09-10T18:49:04.103779+0000 mon.a (mon.0) 3038 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-09-10T18:49:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[18597]: audit 2024-09-10T18:49:04.104690+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-116"}]: dispatch 2024-09-10T18:49:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[18597]: audit 2024-09-10T18:49:05.106894+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-116"}]': finished 2024-09-10T18:49:05.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[31607]: audit 2024-09-10T18:49:04.103600+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[31607]: cluster 2024-09-10T18:49:04.103779+0000 mon.a (mon.0) 3038 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-09-10T18:49:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[31607]: audit 2024-09-10T18:49:04.104690+0000 mon.a (mon.0) 3039 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-116"}]: dispatch 2024-09-10T18:49:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:05 smithi100 bash[31607]: audit 2024-09-10T18:49:05.106894+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-116"}]': finished 2024-09-10T18:49:06.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:06 smithi151 bash[22565]: cluster 2024-09-10T18:49:04.119523+0000 mgr.y (mgr.24512) 572 : cluster [DBG] pgmap v938: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:06.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:06 smithi151 bash[22565]: cluster 2024-09-10T18:49:05.107076+0000 mon.a (mon.0) 3041 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-09-10T18:49:06.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:06 smithi151 bash[22565]: audit 2024-09-10T18:49:05.109703+0000 mon.a (mon.0) 3042 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-116", "mode": "writeback"}]: dispatch 2024-09-10T18:49:06.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:06 smithi151 bash[22565]: cluster 2024-09-10T18:49:06.099840+0000 mon.a (mon.0) 3043 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:06.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:06 smithi151 bash[22565]: audit 2024-09-10T18:49:06.108997+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-116", "mode": "writeback"}]': finished 2024-09-10T18:49:06.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:06 smithi151 bash[22565]: cluster 2024-09-10T18:49:06.109133+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-09-10T18:49:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[18597]: cluster 2024-09-10T18:49:04.119523+0000 mgr.y (mgr.24512) 572 : cluster [DBG] pgmap v938: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[18597]: cluster 2024-09-10T18:49:05.107076+0000 mon.a (mon.0) 3041 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-09-10T18:49:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[18597]: audit 2024-09-10T18:49:05.109703+0000 mon.a (mon.0) 3042 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-116", "mode": "writeback"}]: dispatch 2024-09-10T18:49:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[18597]: cluster 2024-09-10T18:49:06.099840+0000 mon.a (mon.0) 3043 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[18597]: audit 2024-09-10T18:49:06.108997+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-116", "mode": "writeback"}]': finished 2024-09-10T18:49:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[18597]: cluster 2024-09-10T18:49:06.109133+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-09-10T18:49:06.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[31607]: cluster 2024-09-10T18:49:04.119523+0000 mgr.y (mgr.24512) 572 : cluster [DBG] pgmap v938: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[31607]: cluster 2024-09-10T18:49:05.107076+0000 mon.a (mon.0) 3041 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-09-10T18:49:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[31607]: audit 2024-09-10T18:49:05.109703+0000 mon.a (mon.0) 3042 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-116", "mode": "writeback"}]: dispatch 2024-09-10T18:49:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[31607]: cluster 2024-09-10T18:49:06.099840+0000 mon.a (mon.0) 3043 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[31607]: audit 2024-09-10T18:49:06.108997+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-116", "mode": "writeback"}]': finished 2024-09-10T18:49:06.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:06 smithi100 bash[31607]: cluster 2024-09-10T18:49:06.109133+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-09-10T18:49:07.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:07 smithi151 bash[22565]: cluster 2024-09-10T18:49:06.120127+0000 mgr.y (mgr.24512) 573 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:07.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:07 smithi151 bash[22565]: audit 2024-09-10T18:49:06.141945+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.4"}]: dispatch 2024-09-10T18:49:07.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:07 smithi151 bash[22565]: audit 2024-09-10T18:49:06.142284+0000 mgr.y (mgr.24512) 574 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.4"}]: dispatch 2024-09-10T18:49:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:07 smithi100 bash[18597]: cluster 2024-09-10T18:49:06.120127+0000 mgr.y (mgr.24512) 573 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:07 smithi100 bash[18597]: audit 2024-09-10T18:49:06.141945+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.4"}]: dispatch 2024-09-10T18:49:07.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:07 smithi100 bash[18597]: audit 2024-09-10T18:49:06.142284+0000 mgr.y (mgr.24512) 574 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.4"}]: dispatch 2024-09-10T18:49:07.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:07 smithi100 bash[31607]: cluster 2024-09-10T18:49:06.120127+0000 mgr.y (mgr.24512) 573 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:07.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:07 smithi100 bash[31607]: audit 2024-09-10T18:49:06.141945+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.4"}]: dispatch 2024-09-10T18:49:07.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:07 smithi100 bash[31607]: audit 2024-09-10T18:49:06.142284+0000 mgr.y (mgr.24512) 574 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.4"}]: dispatch 2024-09-10T18:49:08.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:08 smithi151 bash[22565]: cluster 2024-09-10T18:49:07.589775+0000 osd.1 (osd.1) 5 : cluster [DBG] 244.4 scrub starts 2024-09-10T18:49:08.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:08 smithi151 bash[22565]: cluster 2024-09-10T18:49:07.594137+0000 osd.1 (osd.1) 6 : cluster [DBG] 244.4 scrub ok 2024-09-10T18:49:08.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:08 smithi100 bash[18597]: cluster 2024-09-10T18:49:07.589775+0000 osd.1 (osd.1) 5 : cluster [DBG] 244.4 scrub starts 2024-09-10T18:49:08.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:08 smithi100 bash[18597]: cluster 2024-09-10T18:49:07.594137+0000 osd.1 (osd.1) 6 : cluster [DBG] 244.4 scrub ok 2024-09-10T18:49:08.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:08 smithi100 bash[31607]: cluster 2024-09-10T18:49:07.589775+0000 osd.1 (osd.1) 5 : cluster [DBG] 244.4 scrub starts 2024-09-10T18:49:08.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:08 smithi100 bash[31607]: cluster 2024-09-10T18:49:07.594137+0000 osd.1 (osd.1) 6 : cluster [DBG] 244.4 scrub ok 2024-09-10T18:49:09.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:09 smithi100 bash[18597]: cluster 2024-09-10T18:49:08.120986+0000 mgr.y (mgr.24512) 575 : cluster [DBG] pgmap v942: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1019 B/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-10T18:49:09.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:09 smithi100 bash[31607]: cluster 2024-09-10T18:49:08.120986+0000 mgr.y (mgr.24512) 575 : cluster [DBG] pgmap v942: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1019 B/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-10T18:49:09.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:09 smithi151 bash[22565]: cluster 2024-09-10T18:49:08.120986+0000 mgr.y (mgr.24512) 575 : cluster [DBG] pgmap v942: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1019 B/s rd, 1.4 KiB/s wr, 2 op/s 2024-09-10T18:49:10.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:10 smithi151 bash[22565]: audit 2024-09-10T18:49:09.347448+0000 mon.a (mon.0) 3047 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:10.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:10 smithi151 bash[22565]: audit 2024-09-10T18:49:09.352194+0000 mon.c (mon.2) 504 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:10 smithi100 bash[18597]: audit 2024-09-10T18:49:09.347448+0000 mon.a (mon.0) 3047 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:10 smithi100 bash[18597]: audit 2024-09-10T18:49:09.352194+0000 mon.c (mon.2) 504 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:10 smithi100 bash[31607]: audit 2024-09-10T18:49:09.347448+0000 mon.a (mon.0) 3047 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:10 smithi100 bash[31607]: audit 2024-09-10T18:49:09.352194+0000 mon.c (mon.2) 504 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:11 smithi151 bash[22565]: cluster 2024-09-10T18:49:10.121779+0000 mgr.y (mgr.24512) 576 : cluster [DBG] pgmap v943: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 850 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:49:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:11 smithi100 bash[18597]: cluster 2024-09-10T18:49:10.121779+0000 mgr.y (mgr.24512) 576 : cluster [DBG] pgmap v943: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 850 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:49:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:49:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:11] "GET /metrics HTTP/1.1" 200 228715 "" "Prometheus/2.43.0" 2024-09-10T18:49:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:11 smithi100 bash[31607]: cluster 2024-09-10T18:49:10.121779+0000 mgr.y (mgr.24512) 576 : cluster [DBG] pgmap v943: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 850 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:49:13.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:49:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:49:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:13 smithi151 bash[22565]: cluster 2024-09-10T18:49:12.123829+0000 mgr.y (mgr.24512) 577 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-09-10T18:49:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:13 smithi151 bash[22565]: audit 2024-09-10T18:49:12.716438+0000 mgr.y (mgr.24512) 578 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:13 smithi100 bash[18597]: cluster 2024-09-10T18:49:12.123829+0000 mgr.y (mgr.24512) 577 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-09-10T18:49:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:13 smithi100 bash[18597]: audit 2024-09-10T18:49:12.716438+0000 mgr.y (mgr.24512) 578 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:13 smithi100 bash[31607]: cluster 2024-09-10T18:49:12.123829+0000 mgr.y (mgr.24512) 577 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-09-10T18:49:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:13 smithi100 bash[31607]: audit 2024-09-10T18:49:12.716438+0000 mgr.y (mgr.24512) 578 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:14.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:49:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:15.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:15 smithi151 bash[22565]: cluster 2024-09-10T18:49:14.124631+0000 mgr.y (mgr.24512) 579 : cluster [DBG] pgmap v945: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 794 B/s wr, 2 op/s 2024-09-10T18:49:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:15 smithi100 bash[18597]: cluster 2024-09-10T18:49:14.124631+0000 mgr.y (mgr.24512) 579 : cluster [DBG] pgmap v945: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 794 B/s wr, 2 op/s 2024-09-10T18:49:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:15 smithi100 bash[31607]: cluster 2024-09-10T18:49:14.124631+0000 mgr.y (mgr.24512) 579 : cluster [DBG] pgmap v945: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 794 B/s wr, 2 op/s 2024-09-10T18:49:17.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:17 smithi151 bash[22565]: cluster 2024-09-10T18:49:16.126708+0000 mgr.y (mgr.24512) 580 : cluster [DBG] pgmap v946: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:17.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:17 smithi100 bash[18597]: cluster 2024-09-10T18:49:16.126708+0000 mgr.y (mgr.24512) 580 : cluster [DBG] pgmap v946: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:17 smithi100 bash[31607]: cluster 2024-09-10T18:49:16.126708+0000 mgr.y (mgr.24512) 580 : cluster [DBG] pgmap v946: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:19.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:19 smithi151 bash[22565]: cluster 2024-09-10T18:49:18.127495+0000 mgr.y (mgr.24512) 581 : cluster [DBG] pgmap v947: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:49:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:19 smithi100 bash[18597]: cluster 2024-09-10T18:49:18.127495+0000 mgr.y (mgr.24512) 581 : cluster [DBG] pgmap v947: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:49:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:19 smithi100 bash[31607]: cluster 2024-09-10T18:49:18.127495+0000 mgr.y (mgr.24512) 581 : cluster [DBG] pgmap v947: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:49:21.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:21 smithi151 bash[22565]: cluster 2024-09-10T18:49:20.128350+0000 mgr.y (mgr.24512) 582 : cluster [DBG] pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:49:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:21 smithi100 bash[18597]: cluster 2024-09-10T18:49:20.128350+0000 mgr.y (mgr.24512) 582 : cluster [DBG] pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:49:21.795 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:49:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:21] "GET /metrics HTTP/1.1" 200 228715 "" "Prometheus/2.43.0" 2024-09-10T18:49:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:21 smithi100 bash[31607]: cluster 2024-09-10T18:49:20.128350+0000 mgr.y (mgr.24512) 582 : cluster [DBG] pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:49:22.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:22 smithi151 bash[22565]: cluster 2024-09-10T18:49:21.391863+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-09-10T18:49:22.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:22 smithi151 bash[22565]: audit 2024-09-10T18:49:21.442747+0000 mon.a (mon.0) 3049 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:22 smithi100 bash[18597]: cluster 2024-09-10T18:49:21.391863+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-09-10T18:49:22.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:22 smithi100 bash[18597]: audit 2024-09-10T18:49:21.442747+0000 mon.a (mon.0) 3049 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:22 smithi100 bash[31607]: cluster 2024-09-10T18:49:21.391863+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-09-10T18:49:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:22 smithi100 bash[31607]: audit 2024-09-10T18:49:21.442747+0000 mon.a (mon.0) 3049 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:23.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:49:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:49:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: cluster 2024-09-10T18:49:22.129272+0000 mgr.y (mgr.24512) 583 : cluster [DBG] pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:49:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: audit 2024-09-10T18:49:22.399736+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: cluster 2024-09-10T18:49:22.399845+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-09-10T18:49:23.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: audit 2024-09-10T18:49:22.401723+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116"}]: dispatch 2024-09-10T18:49:23.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: audit 2024-09-10T18:49:22.724983+0000 mgr.y (mgr.24512) 584 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:23.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: cluster 2024-09-10T18:49:23.390963+0000 mon.a (mon.0) 3053 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:23.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: audit 2024-09-10T18:49:23.397888+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116"}]': finished 2024-09-10T18:49:23.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:23 smithi151 bash[22565]: cluster 2024-09-10T18:49:23.398108+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-09-10T18:49:23.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: cluster 2024-09-10T18:49:22.129272+0000 mgr.y (mgr.24512) 583 : cluster [DBG] pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: audit 2024-09-10T18:49:22.399736+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: cluster 2024-09-10T18:49:22.399845+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: audit 2024-09-10T18:49:22.401723+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116"}]: dispatch 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: audit 2024-09-10T18:49:22.724983+0000 mgr.y (mgr.24512) 584 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: cluster 2024-09-10T18:49:23.390963+0000 mon.a (mon.0) 3053 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: audit 2024-09-10T18:49:23.397888+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116"}]': finished 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[18597]: cluster 2024-09-10T18:49:23.398108+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-09-10T18:49:23.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: cluster 2024-09-10T18:49:22.129272+0000 mgr.y (mgr.24512) 583 : cluster [DBG] pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:49:23.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: audit 2024-09-10T18:49:22.399736+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:23.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: cluster 2024-09-10T18:49:22.399845+0000 mon.a (mon.0) 3051 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-09-10T18:49:23.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: audit 2024-09-10T18:49:22.401723+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116"}]: dispatch 2024-09-10T18:49:23.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: audit 2024-09-10T18:49:22.724983+0000 mgr.y (mgr.24512) 584 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:23.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: cluster 2024-09-10T18:49:23.390963+0000 mon.a (mon.0) 3053 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:23.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: audit 2024-09-10T18:49:23.397888+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-116"}]': finished 2024-09-10T18:49:23.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:23 smithi100 bash[31607]: cluster 2024-09-10T18:49:23.398108+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-09-10T18:49:24.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:49:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:25 smithi151 bash[22565]: cluster 2024-09-10T18:49:24.130120+0000 mgr.y (mgr.24512) 585 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:49:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:25 smithi151 bash[22565]: audit 2024-09-10T18:49:24.345751+0000 mon.a (mon.0) 3056 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:25 smithi151 bash[22565]: audit 2024-09-10T18:49:24.350261+0000 mon.c (mon.2) 505 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:25 smithi151 bash[22565]: cluster 2024-09-10T18:49:24.409082+0000 mon.a (mon.0) 3057 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:25.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:25 smithi151 bash[22565]: cluster 2024-09-10T18:49:24.428156+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-09-10T18:49:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[18597]: cluster 2024-09-10T18:49:24.130120+0000 mgr.y (mgr.24512) 585 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:49:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[18597]: audit 2024-09-10T18:49:24.345751+0000 mon.a (mon.0) 3056 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[18597]: audit 2024-09-10T18:49:24.350261+0000 mon.c (mon.2) 505 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[18597]: cluster 2024-09-10T18:49:24.409082+0000 mon.a (mon.0) 3057 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[18597]: cluster 2024-09-10T18:49:24.428156+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-09-10T18:49:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[31607]: cluster 2024-09-10T18:49:24.130120+0000 mgr.y (mgr.24512) 585 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:49:25.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[31607]: audit 2024-09-10T18:49:24.345751+0000 mon.a (mon.0) 3056 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:25.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[31607]: audit 2024-09-10T18:49:24.350261+0000 mon.c (mon.2) 505 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:25.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[31607]: cluster 2024-09-10T18:49:24.409082+0000 mon.a (mon.0) 3057 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:25.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:25 smithi100 bash[31607]: cluster 2024-09-10T18:49:24.428156+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-09-10T18:49:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:26 smithi100 bash[18597]: cluster 2024-09-10T18:49:25.432630+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-09-10T18:49:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:26 smithi100 bash[18597]: audit 2024-09-10T18:49:25.436117+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:26 smithi100 bash[31607]: cluster 2024-09-10T18:49:25.432630+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-09-10T18:49:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:26 smithi100 bash[31607]: audit 2024-09-10T18:49:25.436117+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:26.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:26 smithi151 bash[22565]: cluster 2024-09-10T18:49:25.432630+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-09-10T18:49:26.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:26 smithi151 bash[22565]: audit 2024-09-10T18:49:25.436117+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:27 smithi100 bash[18597]: cluster 2024-09-10T18:49:26.130944+0000 mgr.y (mgr.24512) 586 : cluster [DBG] pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:49:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:27 smithi100 bash[18597]: audit 2024-09-10T18:49:26.435385+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:27 smithi100 bash[18597]: cluster 2024-09-10T18:49:26.435588+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-09-10T18:49:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:27 smithi100 bash[31607]: cluster 2024-09-10T18:49:26.130944+0000 mgr.y (mgr.24512) 586 : cluster [DBG] pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:49:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:27 smithi100 bash[31607]: audit 2024-09-10T18:49:26.435385+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:27 smithi100 bash[31607]: cluster 2024-09-10T18:49:26.435588+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-09-10T18:49:27.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:27 smithi151 bash[22565]: cluster 2024-09-10T18:49:26.130944+0000 mgr.y (mgr.24512) 586 : cluster [DBG] pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:49:27.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:27 smithi151 bash[22565]: audit 2024-09-10T18:49:26.435385+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:27.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:27 smithi151 bash[22565]: cluster 2024-09-10T18:49:26.435588+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-09-10T18:49:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:28 smithi100 bash[18597]: cluster 2024-09-10T18:49:27.461134+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-09-10T18:49:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:28 smithi100 bash[18597]: audit 2024-09-10T18:49:27.469418+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:28 smithi100 bash[31607]: cluster 2024-09-10T18:49:27.461134+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-09-10T18:49:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:28 smithi100 bash[31607]: audit 2024-09-10T18:49:27.469418+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:28.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:28 smithi151 bash[22565]: cluster 2024-09-10T18:49:27.461134+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-09-10T18:49:28.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:28 smithi151 bash[22565]: audit 2024-09-10T18:49:27.469418+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[18597]: cluster 2024-09-10T18:49:28.131770+0000 mgr.y (mgr.24512) 587 : cluster [DBG] pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:49:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[18597]: audit 2024-09-10T18:49:28.468015+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[18597]: cluster 2024-09-10T18:49:28.468235+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-09-10T18:49:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[18597]: audit 2024-09-10T18:49:28.470968+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-118"}]: dispatch 2024-09-10T18:49:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[18597]: audit 2024-09-10T18:49:29.469211+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-118"}]': finished 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[18597]: cluster 2024-09-10T18:49:29.469762+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[18597]: audit 2024-09-10T18:49:29.471318+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-118", "mode": "writeback"}]: dispatch 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[31607]: cluster 2024-09-10T18:49:28.131770+0000 mgr.y (mgr.24512) 587 : cluster [DBG] pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[31607]: audit 2024-09-10T18:49:28.468015+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[31607]: cluster 2024-09-10T18:49:28.468235+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[31607]: audit 2024-09-10T18:49:28.470968+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-118"}]: dispatch 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[31607]: audit 2024-09-10T18:49:29.469211+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-118"}]': finished 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[31607]: cluster 2024-09-10T18:49:29.469762+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-09-10T18:49:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:29 smithi100 bash[31607]: audit 2024-09-10T18:49:29.471318+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-118", "mode": "writeback"}]: dispatch 2024-09-10T18:49:29.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:29 smithi151 bash[22565]: cluster 2024-09-10T18:49:28.131770+0000 mgr.y (mgr.24512) 587 : cluster [DBG] pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:49:29.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:29 smithi151 bash[22565]: audit 2024-09-10T18:49:28.468015+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:29.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:29 smithi151 bash[22565]: cluster 2024-09-10T18:49:28.468235+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-09-10T18:49:29.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:29 smithi151 bash[22565]: audit 2024-09-10T18:49:28.470968+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-118"}]: dispatch 2024-09-10T18:49:29.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:29 smithi151 bash[22565]: audit 2024-09-10T18:49:29.469211+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-118"}]': finished 2024-09-10T18:49:29.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:29 smithi151 bash[22565]: cluster 2024-09-10T18:49:29.469762+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-09-10T18:49:29.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:29 smithi151 bash[22565]: audit 2024-09-10T18:49:29.471318+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-118", "mode": "writeback"}]: dispatch 2024-09-10T18:49:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:30 smithi100 bash[18597]: cluster 2024-09-10T18:49:30.462926+0000 mon.a (mon.0) 3071 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:30 smithi100 bash[18597]: audit 2024-09-10T18:49:30.471162+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-118", "mode": "writeback"}]': finished 2024-09-10T18:49:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:30 smithi100 bash[18597]: cluster 2024-09-10T18:49:30.471272+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-09-10T18:49:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:30 smithi100 bash[31607]: cluster 2024-09-10T18:49:30.462926+0000 mon.a (mon.0) 3071 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:30 smithi100 bash[31607]: audit 2024-09-10T18:49:30.471162+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-118", "mode": "writeback"}]': finished 2024-09-10T18:49:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:30 smithi100 bash[31607]: cluster 2024-09-10T18:49:30.471272+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-09-10T18:49:30.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:30 smithi151 bash[22565]: cluster 2024-09-10T18:49:30.462926+0000 mon.a (mon.0) 3071 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:30.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:30 smithi151 bash[22565]: audit 2024-09-10T18:49:30.471162+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-118", "mode": "writeback"}]': finished 2024-09-10T18:49:30.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:30 smithi151 bash[22565]: cluster 2024-09-10T18:49:30.471272+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-09-10T18:49:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[18597]: cluster 2024-09-10T18:49:30.132589+0000 mgr.y (mgr.24512) 588 : cluster [DBG] pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[18597]: cluster 2024-09-10T18:49:31.096286+0000 mon.a (mon.0) 3074 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[18597]: cluster 2024-09-10T18:49:31.105918+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-09-10T18:49:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[18597]: audit 2024-09-10T18:49:31.144014+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[31607]: cluster 2024-09-10T18:49:30.132589+0000 mgr.y (mgr.24512) 588 : cluster [DBG] pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[31607]: cluster 2024-09-10T18:49:31.096286+0000 mon.a (mon.0) 3074 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[31607]: cluster 2024-09-10T18:49:31.105918+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-09-10T18:49:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[31607]: audit 2024-09-10T18:49:31.144014+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:31.796 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:49:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:31] "GET /metrics HTTP/1.1" 200 228659 "" "Prometheus/2.43.0" 2024-09-10T18:49:31.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:31 smithi151 bash[22565]: cluster 2024-09-10T18:49:30.132589+0000 mgr.y (mgr.24512) 588 : cluster [DBG] pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:31.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:31 smithi151 bash[22565]: cluster 2024-09-10T18:49:31.096286+0000 mon.a (mon.0) 3074 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:31.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:31 smithi151 bash[22565]: cluster 2024-09-10T18:49:31.105918+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-09-10T18:49:31.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:31 smithi151 bash[22565]: audit 2024-09-10T18:49:31.144014+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:33.115 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:49:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:49:33.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:33 smithi151 bash[22565]: audit 2024-09-10T18:49:32.108341+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:33.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:33 smithi151 bash[22565]: cluster 2024-09-10T18:49:32.108494+0000 mon.a (mon.0) 3078 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-09-10T18:49:33.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:33 smithi151 bash[22565]: audit 2024-09-10T18:49:32.110310+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118"}]: dispatch 2024-09-10T18:49:33.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:33 smithi100 bash[18597]: audit 2024-09-10T18:49:32.108341+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:33 smithi100 bash[18597]: cluster 2024-09-10T18:49:32.108494+0000 mon.a (mon.0) 3078 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-09-10T18:49:33.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:33 smithi100 bash[18597]: audit 2024-09-10T18:49:32.110310+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118"}]: dispatch 2024-09-10T18:49:33.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:33 smithi100 bash[31607]: audit 2024-09-10T18:49:32.108341+0000 mon.a (mon.0) 3077 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:33.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:33 smithi100 bash[31607]: cluster 2024-09-10T18:49:32.108494+0000 mon.a (mon.0) 3078 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-09-10T18:49:33.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:33 smithi100 bash[31607]: audit 2024-09-10T18:49:32.110310+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118"}]: dispatch 2024-09-10T18:49:34.424 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:34 smithi151 bash[22565]: cluster 2024-09-10T18:49:32.133291+0000 mgr.y (mgr.24512) 589 : cluster [DBG] pgmap v966: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 558 B/s wr, 1 op/s 2024-09-10T18:49:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:34 smithi151 bash[22565]: audit 2024-09-10T18:49:32.735484+0000 mgr.y (mgr.24512) 590 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:34 smithi151 bash[22565]: cluster 2024-09-10T18:49:33.103532+0000 mon.a (mon.0) 3080 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:34 smithi151 bash[22565]: audit 2024-09-10T18:49:33.121306+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118"}]': finished 2024-09-10T18:49:34.425 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:34 smithi151 bash[22565]: cluster 2024-09-10T18:49:33.121455+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-09-10T18:49:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[18597]: cluster 2024-09-10T18:49:32.133291+0000 mgr.y (mgr.24512) 589 : cluster [DBG] pgmap v966: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 558 B/s wr, 1 op/s 2024-09-10T18:49:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[18597]: audit 2024-09-10T18:49:32.735484+0000 mgr.y (mgr.24512) 590 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[18597]: cluster 2024-09-10T18:49:33.103532+0000 mon.a (mon.0) 3080 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[18597]: audit 2024-09-10T18:49:33.121306+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118"}]': finished 2024-09-10T18:49:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[18597]: cluster 2024-09-10T18:49:33.121455+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-09-10T18:49:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[31607]: cluster 2024-09-10T18:49:32.133291+0000 mgr.y (mgr.24512) 589 : cluster [DBG] pgmap v966: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 558 B/s wr, 1 op/s 2024-09-10T18:49:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[31607]: audit 2024-09-10T18:49:32.735484+0000 mgr.y (mgr.24512) 590 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:34.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[31607]: cluster 2024-09-10T18:49:33.103532+0000 mon.a (mon.0) 3080 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:34.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[31607]: audit 2024-09-10T18:49:33.121306+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-118"}]': finished 2024-09-10T18:49:34.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:34 smithi100 bash[31607]: cluster 2024-09-10T18:49:33.121455+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-09-10T18:49:34.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:49:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:35.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:35 smithi151 bash[22565]: cluster 2024-09-10T18:49:34.132896+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-09-10T18:49:35.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:35 smithi151 bash[22565]: cluster 2024-09-10T18:49:34.133933+0000 mgr.y (mgr.24512) 591 : cluster [DBG] pgmap v969: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:35.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:35 smithi100 bash[18597]: cluster 2024-09-10T18:49:34.132896+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-09-10T18:49:35.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:35 smithi100 bash[18597]: cluster 2024-09-10T18:49:34.133933+0000 mgr.y (mgr.24512) 591 : cluster [DBG] pgmap v969: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:35 smithi100 bash[31607]: cluster 2024-09-10T18:49:34.132896+0000 mon.a (mon.0) 3083 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-09-10T18:49:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:35 smithi100 bash[31607]: cluster 2024-09-10T18:49:34.133933+0000 mgr.y (mgr.24512) 591 : cluster [DBG] pgmap v969: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:36.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:36 smithi151 bash[22565]: cluster 2024-09-10T18:49:35.143421+0000 mon.a (mon.0) 3084 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-09-10T18:49:36.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:36 smithi151 bash[22565]: audit 2024-09-10T18:49:35.145353+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:36 smithi100 bash[31607]: cluster 2024-09-10T18:49:35.143421+0000 mon.a (mon.0) 3084 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-09-10T18:49:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:36 smithi100 bash[31607]: audit 2024-09-10T18:49:35.145353+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:36 smithi100 bash[18597]: cluster 2024-09-10T18:49:35.143421+0000 mon.a (mon.0) 3084 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-09-10T18:49:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:36 smithi100 bash[18597]: audit 2024-09-10T18:49:35.145353+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:37.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:37 smithi151 bash[22565]: cluster 2024-09-10T18:49:36.135934+0000 mgr.y (mgr.24512) 592 : cluster [DBG] pgmap v971: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:37.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:37 smithi151 bash[22565]: audit 2024-09-10T18:49:36.147367+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:37.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:37 smithi151 bash[22565]: cluster 2024-09-10T18:49:36.147873+0000 mon.a (mon.0) 3087 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-09-10T18:49:37.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:37 smithi151 bash[22565]: audit 2024-09-10T18:49:36.168372+0000 mon.a (mon.0) 3088 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:37.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:37 smithi151 bash[22565]: cluster 2024-09-10T18:49:36.188574+0000 mon.a (mon.0) 3089 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[31607]: cluster 2024-09-10T18:49:36.135934+0000 mgr.y (mgr.24512) 592 : cluster [DBG] pgmap v971: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[31607]: audit 2024-09-10T18:49:36.147367+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[31607]: cluster 2024-09-10T18:49:36.147873+0000 mon.a (mon.0) 3087 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-09-10T18:49:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[31607]: audit 2024-09-10T18:49:36.168372+0000 mon.a (mon.0) 3088 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:37.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[31607]: cluster 2024-09-10T18:49:36.188574+0000 mon.a (mon.0) 3089 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:37.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[18597]: cluster 2024-09-10T18:49:36.135934+0000 mgr.y (mgr.24512) 592 : cluster [DBG] pgmap v971: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:37.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[18597]: audit 2024-09-10T18:49:36.147367+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:37.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[18597]: cluster 2024-09-10T18:49:36.147873+0000 mon.a (mon.0) 3087 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-09-10T18:49:37.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[18597]: audit 2024-09-10T18:49:36.168372+0000 mon.a (mon.0) 3088 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:37.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:37 smithi100 bash[18597]: cluster 2024-09-10T18:49:36.188574+0000 mon.a (mon.0) 3089 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:38 smithi100 bash[31607]: audit 2024-09-10T18:49:37.176058+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:38 smithi100 bash[31607]: cluster 2024-09-10T18:49:37.176197+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-09-10T18:49:38.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:38 smithi100 bash[31607]: audit 2024-09-10T18:49:37.177602+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-120"}]: dispatch 2024-09-10T18:49:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:38 smithi100 bash[18597]: audit 2024-09-10T18:49:37.176058+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:38 smithi100 bash[18597]: cluster 2024-09-10T18:49:37.176197+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-09-10T18:49:38.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:38 smithi100 bash[18597]: audit 2024-09-10T18:49:37.177602+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-120"}]: dispatch 2024-09-10T18:49:38.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:38 smithi151 bash[22565]: audit 2024-09-10T18:49:37.176058+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:38.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:38 smithi151 bash[22565]: cluster 2024-09-10T18:49:37.176197+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-09-10T18:49:38.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:38 smithi151 bash[22565]: audit 2024-09-10T18:49:37.177602+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-120"}]: dispatch 2024-09-10T18:49:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[31607]: cluster 2024-09-10T18:49:38.136781+0000 mgr.y (mgr.24512) 593 : cluster [DBG] pgmap v974: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[31607]: audit 2024-09-10T18:49:38.180290+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-120"}]': finished 2024-09-10T18:49:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[31607]: cluster 2024-09-10T18:49:38.180378+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-09-10T18:49:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[31607]: audit 2024-09-10T18:49:38.182298+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-120", "mode": "writeback"}]: dispatch 2024-09-10T18:49:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[31607]: cluster 2024-09-10T18:49:39.174745+0000 mon.a (mon.0) 3096 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:39.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[31607]: audit 2024-09-10T18:49:39.183909+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-120", "mode": "writeback"}]': finished 2024-09-10T18:49:39.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[18597]: cluster 2024-09-10T18:49:38.136781+0000 mgr.y (mgr.24512) 593 : cluster [DBG] pgmap v974: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:39.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[18597]: audit 2024-09-10T18:49:38.180290+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-120"}]': finished 2024-09-10T18:49:39.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[18597]: cluster 2024-09-10T18:49:38.180378+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-09-10T18:49:39.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[18597]: audit 2024-09-10T18:49:38.182298+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-120", "mode": "writeback"}]: dispatch 2024-09-10T18:49:39.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[18597]: cluster 2024-09-10T18:49:39.174745+0000 mon.a (mon.0) 3096 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:39.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:39 smithi100 bash[18597]: audit 2024-09-10T18:49:39.183909+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-120", "mode": "writeback"}]': finished 2024-09-10T18:49:39.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:39 smithi151 bash[22565]: cluster 2024-09-10T18:49:38.136781+0000 mgr.y (mgr.24512) 593 : cluster [DBG] pgmap v974: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:39.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:39 smithi151 bash[22565]: audit 2024-09-10T18:49:38.180290+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-120"}]': finished 2024-09-10T18:49:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:39 smithi151 bash[22565]: cluster 2024-09-10T18:49:38.180378+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-09-10T18:49:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:39 smithi151 bash[22565]: audit 2024-09-10T18:49:38.182298+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-120", "mode": "writeback"}]: dispatch 2024-09-10T18:49:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:39 smithi151 bash[22565]: cluster 2024-09-10T18:49:39.174745+0000 mon.a (mon.0) 3096 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:39 smithi151 bash[22565]: audit 2024-09-10T18:49:39.183909+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-120", "mode": "writeback"}]': finished 2024-09-10T18:49:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[31607]: cluster 2024-09-10T18:49:39.184020+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-09-10T18:49:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[31607]: audit 2024-09-10T18:49:39.299099+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[31607]: audit 2024-09-10T18:49:39.346352+0000 mon.a (mon.0) 3100 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:40.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[31607]: audit 2024-09-10T18:49:39.351207+0000 mon.c (mon.2) 506 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:40.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[18597]: cluster 2024-09-10T18:49:39.184020+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-09-10T18:49:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[18597]: audit 2024-09-10T18:49:39.299099+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[18597]: audit 2024-09-10T18:49:39.346352+0000 mon.a (mon.0) 3100 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:40.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:40 smithi100 bash[18597]: audit 2024-09-10T18:49:39.351207+0000 mon.c (mon.2) 506 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:40.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:40 smithi151 bash[22565]: cluster 2024-09-10T18:49:39.184020+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-09-10T18:49:40.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:40 smithi151 bash[22565]: audit 2024-09-10T18:49:39.299099+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:40.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:40 smithi151 bash[22565]: audit 2024-09-10T18:49:39.346352+0000 mon.a (mon.0) 3100 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:40.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:40 smithi151 bash[22565]: audit 2024-09-10T18:49:39.351207+0000 mon.c (mon.2) 506 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:41.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:41] "GET /metrics HTTP/1.1" 200 228691 "" "Prometheus/2.43.0" 2024-09-10T18:49:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18597]: cluster 2024-09-10T18:49:40.137676+0000 mgr.y (mgr.24512) 594 : cluster [DBG] pgmap v977: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18597]: audit 2024-09-10T18:49:40.208502+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18597]: cluster 2024-09-10T18:49:40.208672+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-09-10T18:49:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18597]: audit 2024-09-10T18:49:40.209826+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120"}]: dispatch 2024-09-10T18:49:41.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18597]: cluster 2024-09-10T18:49:41.101139+0000 mon.a (mon.0) 3104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18597]: audit 2024-09-10T18:49:41.111596+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120"}]': finished 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[18597]: cluster 2024-09-10T18:49:41.111719+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[31607]: cluster 2024-09-10T18:49:40.137676+0000 mgr.y (mgr.24512) 594 : cluster [DBG] pgmap v977: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[31607]: audit 2024-09-10T18:49:40.208502+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[31607]: cluster 2024-09-10T18:49:40.208672+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[31607]: audit 2024-09-10T18:49:40.209826+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120"}]: dispatch 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[31607]: cluster 2024-09-10T18:49:41.101139+0000 mon.a (mon.0) 3104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[31607]: audit 2024-09-10T18:49:41.111596+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120"}]': finished 2024-09-10T18:49:41.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:41 smithi100 bash[31607]: cluster 2024-09-10T18:49:41.111719+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-09-10T18:49:41.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:41 smithi151 bash[22565]: cluster 2024-09-10T18:49:40.137676+0000 mgr.y (mgr.24512) 594 : cluster [DBG] pgmap v977: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:41.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:41 smithi151 bash[22565]: audit 2024-09-10T18:49:40.208502+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:41.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:41 smithi151 bash[22565]: cluster 2024-09-10T18:49:40.208672+0000 mon.a (mon.0) 3102 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-09-10T18:49:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:41 smithi151 bash[22565]: audit 2024-09-10T18:49:40.209826+0000 mon.a (mon.0) 3103 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120"}]: dispatch 2024-09-10T18:49:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:41 smithi151 bash[22565]: cluster 2024-09-10T18:49:41.101139+0000 mon.a (mon.0) 3104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:41 smithi151 bash[22565]: audit 2024-09-10T18:49:41.111596+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-120"}]': finished 2024-09-10T18:49:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:41 smithi151 bash[22565]: cluster 2024-09-10T18:49:41.111719+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-09-10T18:49:43.124 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:49:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:49:43.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:43 smithi151 bash[22565]: cluster 2024-09-10T18:49:42.113625+0000 mon.a (mon.0) 3107 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-09-10T18:49:43.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:43 smithi100 bash[18597]: cluster 2024-09-10T18:49:42.113625+0000 mon.a (mon.0) 3107 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-09-10T18:49:43.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:43 smithi100 bash[31607]: cluster 2024-09-10T18:49:42.113625+0000 mon.a (mon.0) 3107 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-09-10T18:49:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:44 smithi151 bash[22565]: cluster 2024-09-10T18:49:42.138269+0000 mgr.y (mgr.24512) 595 : cluster [DBG] pgmap v981: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 516 B/s wr, 2 op/s 2024-09-10T18:49:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:44 smithi151 bash[22565]: audit 2024-09-10T18:49:42.743514+0000 mgr.y (mgr.24512) 596 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:44 smithi151 bash[22565]: cluster 2024-09-10T18:49:43.131101+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-09-10T18:49:44.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:44 smithi151 bash[22565]: audit 2024-09-10T18:49:43.136367+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:44.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[18597]: cluster 2024-09-10T18:49:42.138269+0000 mgr.y (mgr.24512) 595 : cluster [DBG] pgmap v981: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 516 B/s wr, 2 op/s 2024-09-10T18:49:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[18597]: audit 2024-09-10T18:49:42.743514+0000 mgr.y (mgr.24512) 596 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[18597]: cluster 2024-09-10T18:49:43.131101+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-09-10T18:49:44.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[18597]: audit 2024-09-10T18:49:43.136367+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[31607]: cluster 2024-09-10T18:49:42.138269+0000 mgr.y (mgr.24512) 595 : cluster [DBG] pgmap v981: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 516 B/s wr, 2 op/s 2024-09-10T18:49:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[31607]: audit 2024-09-10T18:49:42.743514+0000 mgr.y (mgr.24512) 596 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[31607]: cluster 2024-09-10T18:49:43.131101+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-09-10T18:49:44.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:44 smithi100 bash[31607]: audit 2024-09-10T18:49:43.136367+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:44.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:49:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:45 smithi151 bash[22565]: audit 2024-09-10T18:49:44.130387+0000 mon.a (mon.0) 3110 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:45 smithi151 bash[22565]: cluster 2024-09-10T18:49:44.130799+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-09-10T18:49:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:45 smithi151 bash[22565]: cluster 2024-09-10T18:49:44.138847+0000 mgr.y (mgr.24512) 597 : cluster [DBG] pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 2 op/s 2024-09-10T18:49:45.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:45 smithi151 bash[22565]: audit 2024-09-10T18:49:44.152490+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[18597]: audit 2024-09-10T18:49:44.130387+0000 mon.a (mon.0) 3110 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[18597]: cluster 2024-09-10T18:49:44.130799+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[18597]: cluster 2024-09-10T18:49:44.138847+0000 mgr.y (mgr.24512) 597 : cluster [DBG] pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 2 op/s 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[18597]: audit 2024-09-10T18:49:44.152490+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[31607]: audit 2024-09-10T18:49:44.130387+0000 mon.a (mon.0) 3110 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[31607]: cluster 2024-09-10T18:49:44.130799+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[31607]: cluster 2024-09-10T18:49:44.138847+0000 mgr.y (mgr.24512) 597 : cluster [DBG] pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 520 B/s wr, 2 op/s 2024-09-10T18:49:45.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:45 smithi100 bash[31607]: audit 2024-09-10T18:49:44.152490+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:46.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:46 smithi151 bash[22565]: audit 2024-09-10T18:49:45.144220+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:46.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:46 smithi151 bash[22565]: cluster 2024-09-10T18:49:45.144319+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-09-10T18:49:46.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:46 smithi151 bash[22565]: audit 2024-09-10T18:49:45.145831+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-122"}]: dispatch 2024-09-10T18:49:46.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:46 smithi151 bash[22565]: cluster 2024-09-10T18:49:46.102673+0000 mon.a (mon.0) 3116 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:46.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[31607]: audit 2024-09-10T18:49:45.144220+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[31607]: cluster 2024-09-10T18:49:45.144319+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-09-10T18:49:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[31607]: audit 2024-09-10T18:49:45.145831+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-122"}]: dispatch 2024-09-10T18:49:46.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[31607]: cluster 2024-09-10T18:49:46.102673+0000 mon.a (mon.0) 3116 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[18597]: audit 2024-09-10T18:49:45.144220+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[18597]: cluster 2024-09-10T18:49:45.144319+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-09-10T18:49:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[18597]: audit 2024-09-10T18:49:45.145831+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-122"}]: dispatch 2024-09-10T18:49:46.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:46 smithi100 bash[18597]: cluster 2024-09-10T18:49:46.102673+0000 mon.a (mon.0) 3116 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:47.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:47 smithi151 bash[22565]: cluster 2024-09-10T18:49:46.141148+0000 mgr.y (mgr.24512) 598 : cluster [DBG] pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:49:47.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:47 smithi151 bash[22565]: audit 2024-09-10T18:49:46.148804+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-122"}]': finished 2024-09-10T18:49:47.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:47 smithi151 bash[22565]: cluster 2024-09-10T18:49:46.148956+0000 mon.a (mon.0) 3118 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-09-10T18:49:47.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:47 smithi151 bash[22565]: audit 2024-09-10T18:49:46.150937+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-122", "mode": "writeback"}]: dispatch 2024-09-10T18:49:47.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:47 smithi151 bash[22565]: cluster 2024-09-10T18:49:47.142069+0000 mon.a (mon.0) 3120 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:47.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[31607]: cluster 2024-09-10T18:49:46.141148+0000 mgr.y (mgr.24512) 598 : cluster [DBG] pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:49:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[31607]: audit 2024-09-10T18:49:46.148804+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-122"}]': finished 2024-09-10T18:49:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[31607]: cluster 2024-09-10T18:49:46.148956+0000 mon.a (mon.0) 3118 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-09-10T18:49:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[31607]: audit 2024-09-10T18:49:46.150937+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-122", "mode": "writeback"}]: dispatch 2024-09-10T18:49:47.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[31607]: cluster 2024-09-10T18:49:47.142069+0000 mon.a (mon.0) 3120 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[18597]: cluster 2024-09-10T18:49:46.141148+0000 mgr.y (mgr.24512) 598 : cluster [DBG] pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:49:47.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[18597]: audit 2024-09-10T18:49:46.148804+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-122"}]': finished 2024-09-10T18:49:47.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[18597]: cluster 2024-09-10T18:49:46.148956+0000 mon.a (mon.0) 3118 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-09-10T18:49:47.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[18597]: audit 2024-09-10T18:49:46.150937+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-122", "mode": "writeback"}]: dispatch 2024-09-10T18:49:47.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:47 smithi100 bash[18597]: cluster 2024-09-10T18:49:47.142069+0000 mon.a (mon.0) 3120 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:48.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:48 smithi151 bash[22565]: audit 2024-09-10T18:49:47.156171+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-122", "mode": "writeback"}]': finished 2024-09-10T18:49:48.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:48 smithi151 bash[22565]: cluster 2024-09-10T18:49:47.156388+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-09-10T18:49:48.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:48 smithi151 bash[22565]: audit 2024-09-10T18:49:47.256733+0000 mon.a (mon.0) 3123 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:48.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:48 smithi151 bash[22565]: audit 2024-09-10T18:49:47.311924+0000 mon.c (mon.2) 507 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:48.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:48 smithi151 bash[22565]: audit 2024-09-10T18:49:47.767187+0000 mon.c (mon.2) 508 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:48.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:48 smithi151 bash[22565]: audit 2024-09-10T18:49:47.768599+0000 mon.c (mon.2) 509 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:48.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:48 smithi151 bash[22565]: audit 2024-09-10T18:49:47.776659+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[31607]: audit 2024-09-10T18:49:47.156171+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-122", "mode": "writeback"}]': finished 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[31607]: cluster 2024-09-10T18:49:47.156388+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[31607]: audit 2024-09-10T18:49:47.256733+0000 mon.a (mon.0) 3123 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[31607]: audit 2024-09-10T18:49:47.311924+0000 mon.c (mon.2) 507 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[31607]: audit 2024-09-10T18:49:47.767187+0000 mon.c (mon.2) 508 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[31607]: audit 2024-09-10T18:49:47.768599+0000 mon.c (mon.2) 509 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[31607]: audit 2024-09-10T18:49:47.776659+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:48.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[18597]: audit 2024-09-10T18:49:47.156171+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-122", "mode": "writeback"}]': finished 2024-09-10T18:49:48.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[18597]: cluster 2024-09-10T18:49:47.156388+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-09-10T18:49:48.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[18597]: audit 2024-09-10T18:49:47.256733+0000 mon.a (mon.0) 3123 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:48.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[18597]: audit 2024-09-10T18:49:47.311924+0000 mon.c (mon.2) 507 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:48.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[18597]: audit 2024-09-10T18:49:47.767187+0000 mon.c (mon.2) 508 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:48.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[18597]: audit 2024-09-10T18:49:47.768599+0000 mon.c (mon.2) 509 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:48.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:48 smithi100 bash[18597]: audit 2024-09-10T18:49:47.776659+0000 mon.a (mon.0) 3124 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[31607]: cluster 2024-09-10T18:49:48.142013+0000 mgr.y (mgr.24512) 599 : cluster [DBG] pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:49:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[31607]: audit 2024-09-10T18:49:48.180675+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[31607]: cluster 2024-09-10T18:49:48.180781+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-09-10T18:49:49.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[31607]: audit 2024-09-10T18:49:48.182423+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122"}]: dispatch 2024-09-10T18:49:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[18597]: cluster 2024-09-10T18:49:48.142013+0000 mgr.y (mgr.24512) 599 : cluster [DBG] pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:49:49.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[18597]: audit 2024-09-10T18:49:48.180675+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:49.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[18597]: cluster 2024-09-10T18:49:48.180781+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-09-10T18:49:49.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:49 smithi100 bash[18597]: audit 2024-09-10T18:49:48.182423+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122"}]: dispatch 2024-09-10T18:49:49.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:49 smithi151 bash[22565]: cluster 2024-09-10T18:49:48.142013+0000 mgr.y (mgr.24512) 599 : cluster [DBG] pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:49:49.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:49 smithi151 bash[22565]: audit 2024-09-10T18:49:48.180675+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:49.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:49 smithi151 bash[22565]: cluster 2024-09-10T18:49:48.180781+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-09-10T18:49:49.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:49 smithi151 bash[22565]: audit 2024-09-10T18:49:48.182423+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122"}]: dispatch 2024-09-10T18:49:50.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:50 smithi100 bash[18597]: cluster 2024-09-10T18:49:49.177272+0000 mon.a (mon.0) 3128 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:50.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:50 smithi100 bash[18597]: audit 2024-09-10T18:49:49.192192+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122"}]': finished 2024-09-10T18:49:50.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:50 smithi100 bash[18597]: cluster 2024-09-10T18:49:49.192257+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-09-10T18:49:50.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:50 smithi100 bash[31607]: cluster 2024-09-10T18:49:49.177272+0000 mon.a (mon.0) 3128 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:50.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:50 smithi100 bash[31607]: audit 2024-09-10T18:49:49.192192+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122"}]': finished 2024-09-10T18:49:50.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:50 smithi100 bash[31607]: cluster 2024-09-10T18:49:49.192257+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-09-10T18:49:50.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:50 smithi151 bash[22565]: cluster 2024-09-10T18:49:49.177272+0000 mon.a (mon.0) 3128 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:50.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:50 smithi151 bash[22565]: audit 2024-09-10T18:49:49.192192+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-122"}]': finished 2024-09-10T18:49:50.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:50 smithi151 bash[22565]: cluster 2024-09-10T18:49:49.192257+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-09-10T18:49:51.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:49:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:51] "GET /metrics HTTP/1.1" 200 228691 "" "Prometheus/2.43.0" 2024-09-10T18:49:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:51 smithi100 bash[31607]: cluster 2024-09-10T18:49:50.142927+0000 mgr.y (mgr.24512) 600 : cluster [DBG] pgmap v992: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:51 smithi100 bash[31607]: cluster 2024-09-10T18:49:50.204605+0000 mon.a (mon.0) 3131 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-09-10T18:49:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:51 smithi100 bash[31607]: cluster 2024-09-10T18:49:51.104305+0000 mon.a (mon.0) 3132 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:51 smithi100 bash[18597]: cluster 2024-09-10T18:49:50.142927+0000 mgr.y (mgr.24512) 600 : cluster [DBG] pgmap v992: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:51 smithi100 bash[18597]: cluster 2024-09-10T18:49:50.204605+0000 mon.a (mon.0) 3131 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-09-10T18:49:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:51 smithi100 bash[18597]: cluster 2024-09-10T18:49:51.104305+0000 mon.a (mon.0) 3132 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:51.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:51 smithi151 bash[22565]: cluster 2024-09-10T18:49:50.142927+0000 mgr.y (mgr.24512) 600 : cluster [DBG] pgmap v992: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:51.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:51 smithi151 bash[22565]: cluster 2024-09-10T18:49:50.204605+0000 mon.a (mon.0) 3131 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-09-10T18:49:51.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:51 smithi151 bash[22565]: cluster 2024-09-10T18:49:51.104305+0000 mon.a (mon.0) 3132 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:52 smithi100 bash[18597]: cluster 2024-09-10T18:49:51.218795+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-09-10T18:49:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:52 smithi100 bash[18597]: audit 2024-09-10T18:49:51.225186+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:52 smithi100 bash[31607]: cluster 2024-09-10T18:49:51.218795+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-09-10T18:49:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:52 smithi100 bash[31607]: audit 2024-09-10T18:49:51.225186+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:52.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:52 smithi151 bash[22565]: cluster 2024-09-10T18:49:51.218795+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-09-10T18:49:52.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:52 smithi151 bash[22565]: audit 2024-09-10T18:49:51.225186+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:53.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:49:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:49:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[18597]: cluster 2024-09-10T18:49:52.143821+0000 mgr.y (mgr.24512) 601 : cluster [DBG] pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[18597]: audit 2024-09-10T18:49:52.226760+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[18597]: cluster 2024-09-10T18:49:52.227463+0000 mon.a (mon.0) 3136 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-09-10T18:49:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[18597]: audit 2024-09-10T18:49:52.754871+0000 mgr.y (mgr.24512) 602 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[31607]: cluster 2024-09-10T18:49:52.143821+0000 mgr.y (mgr.24512) 601 : cluster [DBG] pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[31607]: audit 2024-09-10T18:49:52.226760+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[31607]: cluster 2024-09-10T18:49:52.227463+0000 mon.a (mon.0) 3136 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-09-10T18:49:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:53 smithi100 bash[31607]: audit 2024-09-10T18:49:52.754871+0000 mgr.y (mgr.24512) 602 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:53.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:53 smithi151 bash[22565]: cluster 2024-09-10T18:49:52.143821+0000 mgr.y (mgr.24512) 601 : cluster [DBG] pgmap v995: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:53.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:53 smithi151 bash[22565]: audit 2024-09-10T18:49:52.226760+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:53.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:53 smithi151 bash[22565]: cluster 2024-09-10T18:49:52.227463+0000 mon.a (mon.0) 3136 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-09-10T18:49:53.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:53 smithi151 bash[22565]: audit 2024-09-10T18:49:52.754871+0000 mgr.y (mgr.24512) 602 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:54.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:54 smithi100 bash[18597]: cluster 2024-09-10T18:49:53.247222+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-09-10T18:49:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:54 smithi100 bash[18597]: audit 2024-09-10T18:49:53.266793+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:54 smithi100 bash[31607]: cluster 2024-09-10T18:49:53.247222+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-09-10T18:49:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:54 smithi100 bash[31607]: audit 2024-09-10T18:49:53.266793+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:54.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:54 smithi151 bash[22565]: cluster 2024-09-10T18:49:53.247222+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-09-10T18:49:54.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:54 smithi151 bash[22565]: audit 2024-09-10T18:49:53.266793+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:54.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:49:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:49:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[18597]: cluster 2024-09-10T18:49:54.144653+0000 mgr.y (mgr.24512) 603 : cluster [DBG] pgmap v998: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[18597]: audit 2024-09-10T18:49:54.259892+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[18597]: cluster 2024-09-10T18:49:54.259962+0000 mon.a (mon.0) 3140 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-09-10T18:49:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[18597]: audit 2024-09-10T18:49:54.262484+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-124"}]: dispatch 2024-09-10T18:49:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[18597]: audit 2024-09-10T18:49:54.345574+0000 mon.a (mon.0) 3142 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:55.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[18597]: audit 2024-09-10T18:49:54.347696+0000 mon.c (mon.2) 510 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[31607]: cluster 2024-09-10T18:49:54.144653+0000 mgr.y (mgr.24512) 603 : cluster [DBG] pgmap v998: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[31607]: audit 2024-09-10T18:49:54.259892+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[31607]: cluster 2024-09-10T18:49:54.259962+0000 mon.a (mon.0) 3140 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-09-10T18:49:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[31607]: audit 2024-09-10T18:49:54.262484+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-124"}]: dispatch 2024-09-10T18:49:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[31607]: audit 2024-09-10T18:49:54.345574+0000 mon.a (mon.0) 3142 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:55 smithi100 bash[31607]: audit 2024-09-10T18:49:54.347696+0000 mon.c (mon.2) 510 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:55 smithi151 bash[22565]: cluster 2024-09-10T18:49:54.144653+0000 mgr.y (mgr.24512) 603 : cluster [DBG] pgmap v998: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:55 smithi151 bash[22565]: audit 2024-09-10T18:49:54.259892+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:55 smithi151 bash[22565]: cluster 2024-09-10T18:49:54.259962+0000 mon.a (mon.0) 3140 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-09-10T18:49:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:55 smithi151 bash[22565]: audit 2024-09-10T18:49:54.262484+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-124"}]: dispatch 2024-09-10T18:49:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:55 smithi151 bash[22565]: audit 2024-09-10T18:49:54.345574+0000 mon.a (mon.0) 3142 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:49:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:55 smithi151 bash[22565]: audit 2024-09-10T18:49:54.347696+0000 mon.c (mon.2) 510 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[18597]: audit 2024-09-10T18:49:55.262913+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-124"}]': finished 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[18597]: cluster 2024-09-10T18:49:55.263026+0000 mon.a (mon.0) 3144 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[18597]: audit 2024-09-10T18:49:55.264958+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-124", "mode": "writeback"}]: dispatch 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[18597]: cluster 2024-09-10T18:49:56.106071+0000 mon.a (mon.0) 3146 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[18597]: cluster 2024-09-10T18:49:56.256472+0000 mon.a (mon.0) 3147 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[31607]: audit 2024-09-10T18:49:55.262913+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-124"}]': finished 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[31607]: cluster 2024-09-10T18:49:55.263026+0000 mon.a (mon.0) 3144 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[31607]: audit 2024-09-10T18:49:55.264958+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-124", "mode": "writeback"}]: dispatch 2024-09-10T18:49:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[31607]: cluster 2024-09-10T18:49:56.106071+0000 mon.a (mon.0) 3146 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:56.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:56 smithi100 bash[31607]: cluster 2024-09-10T18:49:56.256472+0000 mon.a (mon.0) 3147 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:56.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:56 smithi151 bash[22565]: audit 2024-09-10T18:49:55.262913+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-124"}]': finished 2024-09-10T18:49:56.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:56 smithi151 bash[22565]: cluster 2024-09-10T18:49:55.263026+0000 mon.a (mon.0) 3144 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-09-10T18:49:56.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:56 smithi151 bash[22565]: audit 2024-09-10T18:49:55.264958+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-124", "mode": "writeback"}]: dispatch 2024-09-10T18:49:56.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:56 smithi151 bash[22565]: cluster 2024-09-10T18:49:56.106071+0000 mon.a (mon.0) 3146 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:56.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:56 smithi151 bash[22565]: cluster 2024-09-10T18:49:56.256472+0000 mon.a (mon.0) 3147 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:57 smithi100 bash[18597]: cluster 2024-09-10T18:49:56.146670+0000 mgr.y (mgr.24512) 604 : cluster [DBG] pgmap v1001: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:57 smithi100 bash[18597]: audit 2024-09-10T18:49:56.265517+0000 mon.a (mon.0) 3148 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-124", "mode": "writeback"}]': finished 2024-09-10T18:49:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:57 smithi100 bash[18597]: cluster 2024-09-10T18:49:56.265886+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-09-10T18:49:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:57 smithi100 bash[31607]: cluster 2024-09-10T18:49:56.146670+0000 mgr.y (mgr.24512) 604 : cluster [DBG] pgmap v1001: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:57 smithi100 bash[31607]: audit 2024-09-10T18:49:56.265517+0000 mon.a (mon.0) 3148 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-124", "mode": "writeback"}]': finished 2024-09-10T18:49:57.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:57 smithi100 bash[31607]: cluster 2024-09-10T18:49:56.265886+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-09-10T18:49:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:57 smithi151 bash[22565]: cluster 2024-09-10T18:49:56.146670+0000 mgr.y (mgr.24512) 604 : cluster [DBG] pgmap v1001: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:57 smithi151 bash[22565]: audit 2024-09-10T18:49:56.265517+0000 mon.a (mon.0) 3148 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-124", "mode": "writeback"}]': finished 2024-09-10T18:49:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:57 smithi151 bash[22565]: cluster 2024-09-10T18:49:56.265886+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-09-10T18:49:58.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:58 smithi100 bash[18597]: cluster 2024-09-10T18:49:57.288301+0000 mon.a (mon.0) 3150 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-09-10T18:49:58.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:58 smithi100 bash[18597]: audit 2024-09-10T18:49:57.319993+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:58.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:58 smithi100 bash[31607]: cluster 2024-09-10T18:49:57.288301+0000 mon.a (mon.0) 3150 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-09-10T18:49:58.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:58 smithi100 bash[31607]: audit 2024-09-10T18:49:57.319993+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:58 smithi151 bash[22565]: cluster 2024-09-10T18:49:57.288301+0000 mon.a (mon.0) 3150 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-09-10T18:49:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:58 smithi151 bash[22565]: audit 2024-09-10T18:49:57.319993+0000 mon.a (mon.0) 3151 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:49:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:59 smithi151 bash[22565]: cluster 2024-09-10T18:49:58.147607+0000 mgr.y (mgr.24512) 605 : cluster [DBG] pgmap v1004: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:59 smithi151 bash[22565]: audit 2024-09-10T18:49:58.298596+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:59 smithi151 bash[22565]: cluster 2024-09-10T18:49:58.298664+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-09-10T18:49:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:49:59 smithi151 bash[22565]: audit 2024-09-10T18:49:58.299668+0000 mon.a (mon.0) 3154 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124"}]: dispatch 2024-09-10T18:49:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[31607]: cluster 2024-09-10T18:49:58.147607+0000 mgr.y (mgr.24512) 605 : cluster [DBG] pgmap v1004: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[31607]: audit 2024-09-10T18:49:58.298596+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[31607]: cluster 2024-09-10T18:49:58.298664+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-09-10T18:49:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[31607]: audit 2024-09-10T18:49:58.299668+0000 mon.a (mon.0) 3154 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124"}]: dispatch 2024-09-10T18:49:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[18597]: cluster 2024-09-10T18:49:58.147607+0000 mgr.y (mgr.24512) 605 : cluster [DBG] pgmap v1004: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:49:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[18597]: audit 2024-09-10T18:49:58.298596+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:49:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[18597]: cluster 2024-09-10T18:49:58.298664+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-09-10T18:49:59.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:49:59 smithi100 bash[18597]: audit 2024-09-10T18:49:58.299668+0000 mon.a (mon.0) 3154 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124"}]: dispatch 2024-09-10T18:50:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: cluster 2024-09-10T18:49:59.294197+0000 mon.a (mon.0) 3155 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: audit 2024-09-10T18:49:59.310045+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124"}]': finished 2024-09-10T18:50:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: cluster 2024-09-10T18:49:59.310192+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-09-10T18:50:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: cluster 2024-09-10T18:50:00.000163+0000 mon.a (mon.0) 3158 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-10T18:50:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: cluster 2024-09-10T18:50:00.000205+0000 mon.a (mon.0) 3159 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-10T18:50:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: cluster 2024-09-10T18:50:00.000222+0000 mon.a (mon.0) 3160 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:50:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: cluster 2024-09-10T18:50:00.000239+0000 mon.a (mon.0) 3161 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi100-72764-111' 2024-09-10T18:50:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:00 smithi151 bash[22565]: cluster 2024-09-10T18:50:00.000255+0000 mon.a (mon.0) 3162 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: cluster 2024-09-10T18:49:59.294197+0000 mon.a (mon.0) 3155 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: audit 2024-09-10T18:49:59.310045+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124"}]': finished 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: cluster 2024-09-10T18:49:59.310192+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: cluster 2024-09-10T18:50:00.000163+0000 mon.a (mon.0) 3158 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: cluster 2024-09-10T18:50:00.000205+0000 mon.a (mon.0) 3159 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: cluster 2024-09-10T18:50:00.000222+0000 mon.a (mon.0) 3160 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: cluster 2024-09-10T18:50:00.000239+0000 mon.a (mon.0) 3161 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi100-72764-111' 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[31607]: cluster 2024-09-10T18:50:00.000255+0000 mon.a (mon.0) 3162 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:50:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: cluster 2024-09-10T18:49:59.294197+0000 mon.a (mon.0) 3155 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: audit 2024-09-10T18:49:59.310045+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-124"}]': finished 2024-09-10T18:50:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: cluster 2024-09-10T18:49:59.310192+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-09-10T18:50:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: cluster 2024-09-10T18:50:00.000163+0000 mon.a (mon.0) 3158 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-10T18:50:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: cluster 2024-09-10T18:50:00.000205+0000 mon.a (mon.0) 3159 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-10T18:50:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: cluster 2024-09-10T18:50:00.000222+0000 mon.a (mon.0) 3160 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:50:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: cluster 2024-09-10T18:50:00.000239+0000 mon.a (mon.0) 3161 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi100-72764-111' 2024-09-10T18:50:00.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:00 smithi100 bash[18597]: cluster 2024-09-10T18:50:00.000255+0000 mon.a (mon.0) 3162 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:50:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:01 smithi151 bash[22565]: cluster 2024-09-10T18:50:00.148489+0000 mgr.y (mgr.24512) 606 : cluster [DBG] pgmap v1007: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:01 smithi151 bash[22565]: cluster 2024-09-10T18:50:00.327235+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-09-10T18:50:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:01 smithi151 bash[22565]: cluster 2024-09-10T18:50:01.107206+0000 mon.a (mon.0) 3164 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:01.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:50:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:01] "GET /metrics HTTP/1.1" 200 228708 "" "Prometheus/2.43.0" 2024-09-10T18:50:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:01 smithi100 bash[18597]: cluster 2024-09-10T18:50:00.148489+0000 mgr.y (mgr.24512) 606 : cluster [DBG] pgmap v1007: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:01 smithi100 bash[18597]: cluster 2024-09-10T18:50:00.327235+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-09-10T18:50:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:01 smithi100 bash[18597]: cluster 2024-09-10T18:50:01.107206+0000 mon.a (mon.0) 3164 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:01 smithi100 bash[31607]: cluster 2024-09-10T18:50:00.148489+0000 mgr.y (mgr.24512) 606 : cluster [DBG] pgmap v1007: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:01 smithi100 bash[31607]: cluster 2024-09-10T18:50:00.327235+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-09-10T18:50:01.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:01 smithi100 bash[31607]: cluster 2024-09-10T18:50:01.107206+0000 mon.a (mon.0) 3164 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:02 smithi151 bash[22565]: cluster 2024-09-10T18:50:01.344032+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-09-10T18:50:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:02 smithi151 bash[22565]: audit 2024-09-10T18:50:01.346598+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:02 smithi100 bash[18597]: cluster 2024-09-10T18:50:01.344032+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-09-10T18:50:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:02 smithi100 bash[18597]: audit 2024-09-10T18:50:01.346598+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:02 smithi100 bash[31607]: cluster 2024-09-10T18:50:01.344032+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-09-10T18:50:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:02 smithi100 bash[31607]: audit 2024-09-10T18:50:01.346598+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:03.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:50:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:50:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:03 smithi151 bash[22565]: cluster 2024-09-10T18:50:02.149410+0000 mgr.y (mgr.24512) 607 : cluster [DBG] pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:03 smithi151 bash[22565]: audit 2024-09-10T18:50:02.345700+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:03 smithi151 bash[22565]: cluster 2024-09-10T18:50:02.345893+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-09-10T18:50:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:03 smithi151 bash[22565]: audit 2024-09-10T18:50:02.349515+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:03 smithi151 bash[22565]: audit 2024-09-10T18:50:02.766240+0000 mgr.y (mgr.24512) 608 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:03 smithi151 bash[22565]: audit 2024-09-10T18:50:03.345044+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:03 smithi151 bash[22565]: cluster 2024-09-10T18:50:03.345195+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-09-10T18:50:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[18597]: cluster 2024-09-10T18:50:02.149410+0000 mgr.y (mgr.24512) 607 : cluster [DBG] pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[18597]: audit 2024-09-10T18:50:02.345700+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[18597]: cluster 2024-09-10T18:50:02.345893+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-09-10T18:50:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[18597]: audit 2024-09-10T18:50:02.349515+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[18597]: audit 2024-09-10T18:50:02.766240+0000 mgr.y (mgr.24512) 608 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[18597]: audit 2024-09-10T18:50:03.345044+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[18597]: cluster 2024-09-10T18:50:03.345195+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-09-10T18:50:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[31607]: cluster 2024-09-10T18:50:02.149410+0000 mgr.y (mgr.24512) 607 : cluster [DBG] pgmap v1010: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[31607]: audit 2024-09-10T18:50:02.345700+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[31607]: cluster 2024-09-10T18:50:02.345893+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-09-10T18:50:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[31607]: audit 2024-09-10T18:50:02.349515+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[31607]: audit 2024-09-10T18:50:02.766240+0000 mgr.y (mgr.24512) 608 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[31607]: audit 2024-09-10T18:50:03.345044+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:03 smithi100 bash[31607]: cluster 2024-09-10T18:50:03.345195+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-09-10T18:50:04.671 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:50:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:04 smithi151 bash[22565]: audit 2024-09-10T18:50:03.350939+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-126"}]: dispatch 2024-09-10T18:50:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:04 smithi151 bash[22565]: audit 2024-09-10T18:50:04.350305+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-126"}]': finished 2024-09-10T18:50:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:04 smithi151 bash[22565]: cluster 2024-09-10T18:50:04.350490+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-09-10T18:50:04.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:04 smithi151 bash[22565]: audit 2024-09-10T18:50:04.351389+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-126", "mode": "writeback"}]: dispatch 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[18597]: audit 2024-09-10T18:50:03.350939+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-126"}]: dispatch 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[18597]: audit 2024-09-10T18:50:04.350305+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-126"}]': finished 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[18597]: cluster 2024-09-10T18:50:04.350490+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[18597]: audit 2024-09-10T18:50:04.351389+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-126", "mode": "writeback"}]: dispatch 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[31607]: audit 2024-09-10T18:50:03.350939+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-126"}]: dispatch 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[31607]: audit 2024-09-10T18:50:04.350305+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-126"}]': finished 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[31607]: cluster 2024-09-10T18:50:04.350490+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-09-10T18:50:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:04 smithi100 bash[31607]: audit 2024-09-10T18:50:04.351389+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-126", "mode": "writeback"}]: dispatch 2024-09-10T18:50:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:05 smithi151 bash[22565]: cluster 2024-09-10T18:50:04.150230+0000 mgr.y (mgr.24512) 609 : cluster [DBG] pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:05 smithi151 bash[22565]: cluster 2024-09-10T18:50:05.344108+0000 mon.a (mon.0) 3176 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:05 smithi151 bash[22565]: audit 2024-09-10T18:50:05.353073+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-126", "mode": "writeback"}]': finished 2024-09-10T18:50:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:05 smithi151 bash[22565]: cluster 2024-09-10T18:50:05.353169+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[31607]: cluster 2024-09-10T18:50:04.150230+0000 mgr.y (mgr.24512) 609 : cluster [DBG] pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[31607]: cluster 2024-09-10T18:50:05.344108+0000 mon.a (mon.0) 3176 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[31607]: audit 2024-09-10T18:50:05.353073+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-126", "mode": "writeback"}]': finished 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[31607]: cluster 2024-09-10T18:50:05.353169+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[18597]: cluster 2024-09-10T18:50:04.150230+0000 mgr.y (mgr.24512) 609 : cluster [DBG] pgmap v1013: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[18597]: cluster 2024-09-10T18:50:05.344108+0000 mon.a (mon.0) 3176 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[18597]: audit 2024-09-10T18:50:05.353073+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-126", "mode": "writeback"}]': finished 2024-09-10T18:50:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:05 smithi100 bash[18597]: cluster 2024-09-10T18:50:05.353169+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-09-10T18:50:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:06 smithi151 bash[22565]: audit 2024-09-10T18:50:05.460776+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:06 smithi151 bash[22565]: cluster 2024-09-10T18:50:06.113843+0000 mon.a (mon.0) 3180 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:06 smithi151 bash[22565]: audit 2024-09-10T18:50:06.126564+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:06 smithi151 bash[22565]: cluster 2024-09-10T18:50:06.126734+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-09-10T18:50:06.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:06 smithi151 bash[22565]: audit 2024-09-10T18:50:06.128313+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126"}]: dispatch 2024-09-10T18:50:06.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[31607]: audit 2024-09-10T18:50:05.460776+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:06.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[31607]: cluster 2024-09-10T18:50:06.113843+0000 mon.a (mon.0) 3180 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:06.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[31607]: audit 2024-09-10T18:50:06.126564+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:06.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[31607]: cluster 2024-09-10T18:50:06.126734+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-09-10T18:50:06.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[31607]: audit 2024-09-10T18:50:06.128313+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126"}]: dispatch 2024-09-10T18:50:06.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[18597]: audit 2024-09-10T18:50:05.460776+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:06.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[18597]: cluster 2024-09-10T18:50:06.113843+0000 mon.a (mon.0) 3180 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:06.799 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[18597]: audit 2024-09-10T18:50:06.126564+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:06.799 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[18597]: cluster 2024-09-10T18:50:06.126734+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-09-10T18:50:06.799 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:06 smithi100 bash[18597]: audit 2024-09-10T18:50:06.128313+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126"}]: dispatch 2024-09-10T18:50:07.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:07 smithi151 bash[22565]: cluster 2024-09-10T18:50:06.150990+0000 mgr.y (mgr.24512) 610 : cluster [DBG] pgmap v1017: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:07.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:07 smithi151 bash[22565]: cluster 2024-09-10T18:50:07.118330+0000 mon.a (mon.0) 3184 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:07.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:07 smithi151 bash[22565]: audit 2024-09-10T18:50:07.132437+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126"}]': finished 2024-09-10T18:50:07.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:07 smithi151 bash[22565]: cluster 2024-09-10T18:50:07.132519+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-09-10T18:50:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[31607]: cluster 2024-09-10T18:50:06.150990+0000 mgr.y (mgr.24512) 610 : cluster [DBG] pgmap v1017: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[31607]: cluster 2024-09-10T18:50:07.118330+0000 mon.a (mon.0) 3184 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[31607]: audit 2024-09-10T18:50:07.132437+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126"}]': finished 2024-09-10T18:50:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[31607]: cluster 2024-09-10T18:50:07.132519+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-09-10T18:50:07.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[18597]: cluster 2024-09-10T18:50:06.150990+0000 mgr.y (mgr.24512) 610 : cluster [DBG] pgmap v1017: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:07.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[18597]: cluster 2024-09-10T18:50:07.118330+0000 mon.a (mon.0) 3184 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:07.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[18597]: audit 2024-09-10T18:50:07.132437+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-126"}]': finished 2024-09-10T18:50:07.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:07 smithi100 bash[18597]: cluster 2024-09-10T18:50:07.132519+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9051 ms) 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6082 ms) 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5053 ms) 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:50:08.139 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25162 ms) 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14077 ms) 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12105 ms) 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 00000012 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12139 ms) 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-10T18:50:08.140 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6090 ms) 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (559425 ms total) 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1023 ms) 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11101 ms) 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18184 ms) 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30311 ms total) 2024-09-10T18:50:08.141 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7063 ms) 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8099 ms) 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: waiting for scrub... 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: done waiting 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24367 ms) 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9702 ms) 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-10T18:50:08.142 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7981 ms) 2024-09-10T18:50:08.143 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-10T18:50:08.143 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8099 ms) 2024-09-10T18:50:08.143 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-10T18:50:08.143 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10118 ms) 2024-09-10T18:50:08.143 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-10T18:50:09.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:09 smithi151 bash[22565]: cluster 2024-09-10T18:50:08.137569+0000 mon.a (mon.0) 3187 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-09-10T18:50:09.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:09 smithi151 bash[22565]: cluster 2024-09-10T18:50:08.151477+0000 mgr.y (mgr.24512) 611 : cluster [DBG] pgmap v1020: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:09.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:09 smithi100 bash[18597]: cluster 2024-09-10T18:50:08.137569+0000 mon.a (mon.0) 3187 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-09-10T18:50:09.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:09 smithi100 bash[18597]: cluster 2024-09-10T18:50:08.151477+0000 mgr.y (mgr.24512) 611 : cluster [DBG] pgmap v1020: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:09.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:09 smithi100 bash[31607]: cluster 2024-09-10T18:50:08.137569+0000 mon.a (mon.0) 3187 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-09-10T18:50:09.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:09 smithi100 bash[31607]: cluster 2024-09-10T18:50:08.151477+0000 mgr.y (mgr.24512) 611 : cluster [DBG] pgmap v1020: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:10.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:10 smithi151 bash[22565]: cluster 2024-09-10T18:50:09.151975+0000 mon.a (mon.0) 3188 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-09-10T18:50:10.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:10 smithi151 bash[22565]: audit 2024-09-10T18:50:09.155363+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:10.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:10 smithi151 bash[22565]: audit 2024-09-10T18:50:09.340124+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:10 smithi100 bash[18597]: cluster 2024-09-10T18:50:09.151975+0000 mon.a (mon.0) 3188 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-09-10T18:50:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:10 smithi100 bash[18597]: audit 2024-09-10T18:50:09.155363+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:10.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:10 smithi100 bash[18597]: audit 2024-09-10T18:50:09.340124+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:10.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:10 smithi100 bash[31607]: cluster 2024-09-10T18:50:09.151975+0000 mon.a (mon.0) 3188 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-09-10T18:50:10.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:10 smithi100 bash[31607]: audit 2024-09-10T18:50:09.155363+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:10.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:10 smithi100 bash[31607]: audit 2024-09-10T18:50:09.340124+0000 mon.c (mon.2) 511 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:11.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:11] "GET /metrics HTTP/1.1" 200 228701 "" "Prometheus/2.43.0" 2024-09-10T18:50:11.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[18597]: cluster 2024-09-10T18:50:10.152385+0000 mgr.y (mgr.24512) 612 : cluster [DBG] pgmap v1022: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:11.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[18597]: audit 2024-09-10T18:50:10.164567+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:11.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[18597]: cluster 2024-09-10T18:50:10.165074+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-09-10T18:50:11.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[18597]: audit 2024-09-10T18:50:10.168511+0000 mon.a (mon.0) 3192 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:11.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[18597]: cluster 2024-09-10T18:50:11.144928+0000 mon.a (mon.0) 3193 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:11.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[31607]: cluster 2024-09-10T18:50:10.152385+0000 mgr.y (mgr.24512) 612 : cluster [DBG] pgmap v1022: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:11.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[31607]: audit 2024-09-10T18:50:10.164567+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:11.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[31607]: cluster 2024-09-10T18:50:10.165074+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-09-10T18:50:11.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[31607]: audit 2024-09-10T18:50:10.168511+0000 mon.a (mon.0) 3192 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:11.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:11 smithi100 bash[31607]: cluster 2024-09-10T18:50:11.144928+0000 mon.a (mon.0) 3193 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:11 smithi151 bash[22565]: cluster 2024-09-10T18:50:10.152385+0000 mgr.y (mgr.24512) 612 : cluster [DBG] pgmap v1022: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:11 smithi151 bash[22565]: audit 2024-09-10T18:50:10.164567+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:11 smithi151 bash[22565]: cluster 2024-09-10T18:50:10.165074+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-09-10T18:50:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:11 smithi151 bash[22565]: audit 2024-09-10T18:50:10.168511+0000 mon.a (mon.0) 3192 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:11 smithi151 bash[22565]: cluster 2024-09-10T18:50:11.144928+0000 mon.a (mon.0) 3193 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[18597]: audit 2024-09-10T18:50:11.166239+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[18597]: cluster 2024-09-10T18:50:11.166351+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-09-10T18:50:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[18597]: audit 2024-09-10T18:50:11.168827+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-128"}]: dispatch 2024-09-10T18:50:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[18597]: audit 2024-09-10T18:50:12.169430+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-128"}]': finished 2024-09-10T18:50:12.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[18597]: cluster 2024-09-10T18:50:12.169626+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-09-10T18:50:12.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[31607]: audit 2024-09-10T18:50:11.166239+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:12.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[31607]: cluster 2024-09-10T18:50:11.166351+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-09-10T18:50:12.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[31607]: audit 2024-09-10T18:50:11.168827+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-128"}]: dispatch 2024-09-10T18:50:12.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[31607]: audit 2024-09-10T18:50:12.169430+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-128"}]': finished 2024-09-10T18:50:12.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:12 smithi100 bash[31607]: cluster 2024-09-10T18:50:12.169626+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-09-10T18:50:12.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:12 smithi151 bash[22565]: audit 2024-09-10T18:50:11.166239+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:12.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:12 smithi151 bash[22565]: cluster 2024-09-10T18:50:11.166351+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-09-10T18:50:12.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:12 smithi151 bash[22565]: audit 2024-09-10T18:50:11.168827+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-128"}]: dispatch 2024-09-10T18:50:12.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:12 smithi151 bash[22565]: audit 2024-09-10T18:50:12.169430+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-128"}]': finished 2024-09-10T18:50:12.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:12 smithi151 bash[22565]: cluster 2024-09-10T18:50:12.169626+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-09-10T18:50:13.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:50:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:50:13.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[18597]: cluster 2024-09-10T18:50:12.153233+0000 mgr.y (mgr.24512) 613 : cluster [DBG] pgmap v1025: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:50:13.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[18597]: audit 2024-09-10T18:50:12.170505+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-128", "mode": "writeback"}]: dispatch 2024-09-10T18:50:13.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[18597]: audit 2024-09-10T18:50:12.777704+0000 mgr.y (mgr.24512) 614 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:13.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[18597]: cluster 2024-09-10T18:50:13.162985+0000 mon.a (mon.0) 3200 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:13.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[18597]: audit 2024-09-10T18:50:13.175000+0000 mon.a (mon.0) 3201 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-128", "mode": "writeback"}]': finished 2024-09-10T18:50:13.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[18597]: cluster 2024-09-10T18:50:13.175197+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-09-10T18:50:13.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[31607]: cluster 2024-09-10T18:50:12.153233+0000 mgr.y (mgr.24512) 613 : cluster [DBG] pgmap v1025: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:50:13.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[31607]: audit 2024-09-10T18:50:12.170505+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-128", "mode": "writeback"}]: dispatch 2024-09-10T18:50:13.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[31607]: audit 2024-09-10T18:50:12.777704+0000 mgr.y (mgr.24512) 614 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:13.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[31607]: cluster 2024-09-10T18:50:13.162985+0000 mon.a (mon.0) 3200 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:13.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[31607]: audit 2024-09-10T18:50:13.175000+0000 mon.a (mon.0) 3201 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-128", "mode": "writeback"}]': finished 2024-09-10T18:50:13.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:13 smithi100 bash[31607]: cluster 2024-09-10T18:50:13.175197+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-09-10T18:50:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:13 smithi151 bash[22565]: cluster 2024-09-10T18:50:12.153233+0000 mgr.y (mgr.24512) 613 : cluster [DBG] pgmap v1025: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:50:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:13 smithi151 bash[22565]: audit 2024-09-10T18:50:12.170505+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-128", "mode": "writeback"}]: dispatch 2024-09-10T18:50:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:13 smithi151 bash[22565]: audit 2024-09-10T18:50:12.777704+0000 mgr.y (mgr.24512) 614 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:13 smithi151 bash[22565]: cluster 2024-09-10T18:50:13.162985+0000 mon.a (mon.0) 3200 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:13 smithi151 bash[22565]: audit 2024-09-10T18:50:13.175000+0000 mon.a (mon.0) 3201 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-128", "mode": "writeback"}]': finished 2024-09-10T18:50:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:13 smithi151 bash[22565]: cluster 2024-09-10T18:50:13.175197+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-09-10T18:50:14.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:50:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:15.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:15 smithi100 bash[18597]: cluster 2024-09-10T18:50:14.154064+0000 mgr.y (mgr.24512) 615 : cluster [DBG] pgmap v1028: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:15.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:15 smithi100 bash[31607]: cluster 2024-09-10T18:50:14.154064+0000 mgr.y (mgr.24512) 615 : cluster [DBG] pgmap v1028: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:15.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:15 smithi151 bash[22565]: cluster 2024-09-10T18:50:14.154064+0000 mgr.y (mgr.24512) 615 : cluster [DBG] pgmap v1028: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:17.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:17 smithi100 bash[18597]: cluster 2024-09-10T18:50:16.156166+0000 mgr.y (mgr.24512) 616 : cluster [DBG] pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 512 B/s wr, 2 op/s 2024-09-10T18:50:17.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:17 smithi100 bash[31607]: cluster 2024-09-10T18:50:16.156166+0000 mgr.y (mgr.24512) 616 : cluster [DBG] pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 512 B/s wr, 2 op/s 2024-09-10T18:50:17.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:17 smithi151 bash[22565]: cluster 2024-09-10T18:50:16.156166+0000 mgr.y (mgr.24512) 616 : cluster [DBG] pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 512 B/s wr, 2 op/s 2024-09-10T18:50:19.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:19 smithi100 bash[18597]: cluster 2024-09-10T18:50:18.157142+0000 mgr.y (mgr.24512) 617 : cluster [DBG] pgmap v1030: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 878 B/s rd, 292 B/s wr, 1 op/s 2024-09-10T18:50:19.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:19 smithi100 bash[18597]: audit 2024-09-10T18:50:18.289814+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:19.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:19 smithi100 bash[31607]: cluster 2024-09-10T18:50:18.157142+0000 mgr.y (mgr.24512) 617 : cluster [DBG] pgmap v1030: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 878 B/s rd, 292 B/s wr, 1 op/s 2024-09-10T18:50:19.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:19 smithi100 bash[31607]: audit 2024-09-10T18:50:18.289814+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:19.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:19 smithi151 bash[22565]: cluster 2024-09-10T18:50:18.157142+0000 mgr.y (mgr.24512) 617 : cluster [DBG] pgmap v1030: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 878 B/s rd, 292 B/s wr, 1 op/s 2024-09-10T18:50:19.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:19 smithi151 bash[22565]: audit 2024-09-10T18:50:18.289814+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:20 smithi100 bash[18597]: audit 2024-09-10T18:50:19.205213+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:20 smithi100 bash[18597]: cluster 2024-09-10T18:50:19.205369+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-09-10T18:50:20.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:20 smithi100 bash[18597]: audit 2024-09-10T18:50:19.206180+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128"}]: dispatch 2024-09-10T18:50:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:20 smithi100 bash[31607]: audit 2024-09-10T18:50:19.205213+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:20 smithi100 bash[31607]: cluster 2024-09-10T18:50:19.205369+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-09-10T18:50:20.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:20 smithi100 bash[31607]: audit 2024-09-10T18:50:19.206180+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128"}]: dispatch 2024-09-10T18:50:20.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:20 smithi151 bash[22565]: audit 2024-09-10T18:50:19.205213+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:20.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:20 smithi151 bash[22565]: cluster 2024-09-10T18:50:19.205369+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-09-10T18:50:20.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:20 smithi151 bash[22565]: audit 2024-09-10T18:50:19.206180+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128"}]: dispatch 2024-09-10T18:50:21.544 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:21] "GET /metrics HTTP/1.1" 200 228701 "" "Prometheus/2.43.0" 2024-09-10T18:50:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[31607]: cluster 2024-09-10T18:50:20.158023+0000 mgr.y (mgr.24512) 618 : cluster [DBG] pgmap v1032: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 768 B/s rd, 256 B/s wr, 1 op/s 2024-09-10T18:50:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[31607]: cluster 2024-09-10T18:50:20.199989+0000 mon.a (mon.0) 3207 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[31607]: audit 2024-09-10T18:50:20.208945+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128"}]': finished 2024-09-10T18:50:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[31607]: cluster 2024-09-10T18:50:20.209153+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-09-10T18:50:21.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[31607]: cluster 2024-09-10T18:50:21.118713+0000 mon.a (mon.0) 3210 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[18597]: cluster 2024-09-10T18:50:20.158023+0000 mgr.y (mgr.24512) 618 : cluster [DBG] pgmap v1032: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 768 B/s rd, 256 B/s wr, 1 op/s 2024-09-10T18:50:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[18597]: cluster 2024-09-10T18:50:20.199989+0000 mon.a (mon.0) 3207 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[18597]: audit 2024-09-10T18:50:20.208945+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128"}]': finished 2024-09-10T18:50:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[18597]: cluster 2024-09-10T18:50:20.209153+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-09-10T18:50:21.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:21 smithi100 bash[18597]: cluster 2024-09-10T18:50:21.118713+0000 mon.a (mon.0) 3210 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:21.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:21 smithi151 bash[22565]: cluster 2024-09-10T18:50:20.158023+0000 mgr.y (mgr.24512) 618 : cluster [DBG] pgmap v1032: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 768 B/s rd, 256 B/s wr, 1 op/s 2024-09-10T18:50:21.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:21 smithi151 bash[22565]: cluster 2024-09-10T18:50:20.199989+0000 mon.a (mon.0) 3207 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:21.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:21 smithi151 bash[22565]: audit 2024-09-10T18:50:20.208945+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-128"}]': finished 2024-09-10T18:50:21.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:21 smithi151 bash[22565]: cluster 2024-09-10T18:50:20.209153+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-09-10T18:50:21.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:21 smithi151 bash[22565]: cluster 2024-09-10T18:50:21.118713+0000 mon.a (mon.0) 3210 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:22 smithi100 bash[18597]: cluster 2024-09-10T18:50:21.227416+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-09-10T18:50:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:22 smithi100 bash[31607]: cluster 2024-09-10T18:50:21.227416+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-09-10T18:50:22.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:22 smithi151 bash[22565]: cluster 2024-09-10T18:50:21.227416+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-09-10T18:50:23.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:50:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:50:23.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[18597]: cluster 2024-09-10T18:50:22.158827+0000 mgr.y (mgr.24512) 619 : cluster [DBG] pgmap v1035: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:50:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[18597]: cluster 2024-09-10T18:50:22.242810+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-09-10T18:50:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[18597]: audit 2024-09-10T18:50:22.245574+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[18597]: audit 2024-09-10T18:50:22.789170+0000 mgr.y (mgr.24512) 620 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[31607]: cluster 2024-09-10T18:50:22.158827+0000 mgr.y (mgr.24512) 619 : cluster [DBG] pgmap v1035: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:50:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[31607]: cluster 2024-09-10T18:50:22.242810+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-09-10T18:50:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[31607]: audit 2024-09-10T18:50:22.245574+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:23.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:23 smithi100 bash[31607]: audit 2024-09-10T18:50:22.789170+0000 mgr.y (mgr.24512) 620 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:23 smithi151 bash[22565]: cluster 2024-09-10T18:50:22.158827+0000 mgr.y (mgr.24512) 619 : cluster [DBG] pgmap v1035: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T18:50:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:23 smithi151 bash[22565]: cluster 2024-09-10T18:50:22.242810+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-09-10T18:50:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:23 smithi151 bash[22565]: audit 2024-09-10T18:50:22.245574+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:23 smithi151 bash[22565]: audit 2024-09-10T18:50:22.789170+0000 mgr.y (mgr.24512) 620 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[18597]: audit 2024-09-10T18:50:23.246754+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[18597]: cluster 2024-09-10T18:50:23.247222+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-09-10T18:50:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[18597]: audit 2024-09-10T18:50:23.249915+0000 mon.a (mon.0) 3216 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:24.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[18597]: audit 2024-09-10T18:50:24.249694+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[31607]: audit 2024-09-10T18:50:23.246754+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[31607]: cluster 2024-09-10T18:50:23.247222+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-09-10T18:50:24.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[31607]: audit 2024-09-10T18:50:23.249915+0000 mon.a (mon.0) 3216 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:24.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:24 smithi100 bash[31607]: audit 2024-09-10T18:50:24.249694+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:24.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:24 smithi151 bash[22565]: audit 2024-09-10T18:50:23.246754+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:24.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:24 smithi151 bash[22565]: cluster 2024-09-10T18:50:23.247222+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-09-10T18:50:24.547 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:24 smithi151 bash[22565]: audit 2024-09-10T18:50:23.249915+0000 mon.a (mon.0) 3216 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:24.547 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:24 smithi151 bash[22565]: audit 2024-09-10T18:50:24.249694+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:24.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:50:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[18597]: cluster 2024-09-10T18:50:24.159736+0000 mgr.y (mgr.24512) 621 : cluster [DBG] pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:50:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[18597]: cluster 2024-09-10T18:50:24.249855+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-09-10T18:50:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[18597]: audit 2024-09-10T18:50:24.255005+0000 mon.a (mon.0) 3219 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-130"}]: dispatch 2024-09-10T18:50:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[18597]: audit 2024-09-10T18:50:24.340391+0000 mon.c (mon.2) 512 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[18597]: audit 2024-09-10T18:50:25.252592+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-130"}]': finished 2024-09-10T18:50:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[18597]: cluster 2024-09-10T18:50:25.252761+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-09-10T18:50:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[18597]: audit 2024-09-10T18:50:25.254199+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-130", "mode": "writeback"}]: dispatch 2024-09-10T18:50:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[31607]: cluster 2024-09-10T18:50:24.159736+0000 mgr.y (mgr.24512) 621 : cluster [DBG] pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:50:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[31607]: cluster 2024-09-10T18:50:24.249855+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-09-10T18:50:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[31607]: audit 2024-09-10T18:50:24.255005+0000 mon.a (mon.0) 3219 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-130"}]: dispatch 2024-09-10T18:50:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[31607]: audit 2024-09-10T18:50:24.340391+0000 mon.c (mon.2) 512 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[31607]: audit 2024-09-10T18:50:25.252592+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-130"}]': finished 2024-09-10T18:50:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[31607]: cluster 2024-09-10T18:50:25.252761+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-09-10T18:50:25.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:25 smithi100 bash[31607]: audit 2024-09-10T18:50:25.254199+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-130", "mode": "writeback"}]: dispatch 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:25 smithi151 bash[22565]: cluster 2024-09-10T18:50:24.159736+0000 mgr.y (mgr.24512) 621 : cluster [DBG] pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:25 smithi151 bash[22565]: cluster 2024-09-10T18:50:24.249855+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:25 smithi151 bash[22565]: audit 2024-09-10T18:50:24.255005+0000 mon.a (mon.0) 3219 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-130"}]: dispatch 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:25 smithi151 bash[22565]: audit 2024-09-10T18:50:24.340391+0000 mon.c (mon.2) 512 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:25 smithi151 bash[22565]: audit 2024-09-10T18:50:25.252592+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-130"}]': finished 2024-09-10T18:50:25.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:25 smithi151 bash[22565]: cluster 2024-09-10T18:50:25.252761+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-09-10T18:50:25.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:25 smithi151 bash[22565]: audit 2024-09-10T18:50:25.254199+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-130", "mode": "writeback"}]: dispatch 2024-09-10T18:50:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:26 smithi100 bash[18597]: cluster 2024-09-10T18:50:26.247162+0000 mon.a (mon.0) 3223 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:26 smithi100 bash[18597]: audit 2024-09-10T18:50:26.255544+0000 mon.a (mon.0) 3224 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-130", "mode": "writeback"}]': finished 2024-09-10T18:50:26.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:26 smithi100 bash[18597]: cluster 2024-09-10T18:50:26.255699+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-09-10T18:50:26.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:26 smithi100 bash[31607]: cluster 2024-09-10T18:50:26.247162+0000 mon.a (mon.0) 3223 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:26.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:26 smithi100 bash[31607]: audit 2024-09-10T18:50:26.255544+0000 mon.a (mon.0) 3224 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-130", "mode": "writeback"}]': finished 2024-09-10T18:50:26.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:26 smithi100 bash[31607]: cluster 2024-09-10T18:50:26.255699+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-09-10T18:50:26.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:26 smithi151 bash[22565]: cluster 2024-09-10T18:50:26.247162+0000 mon.a (mon.0) 3223 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:26.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:26 smithi151 bash[22565]: audit 2024-09-10T18:50:26.255544+0000 mon.a (mon.0) 3224 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-130", "mode": "writeback"}]': finished 2024-09-10T18:50:26.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:26 smithi151 bash[22565]: cluster 2024-09-10T18:50:26.255699+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-09-10T18:50:27.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:27 smithi100 bash[18597]: cluster 2024-09-10T18:50:26.161796+0000 mgr.y (mgr.24512) 622 : cluster [DBG] pgmap v1041: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:27.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:27 smithi100 bash[18597]: audit 2024-09-10T18:50:26.421081+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:27.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:27 smithi100 bash[31607]: cluster 2024-09-10T18:50:26.161796+0000 mgr.y (mgr.24512) 622 : cluster [DBG] pgmap v1041: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:27.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:27 smithi100 bash[31607]: audit 2024-09-10T18:50:26.421081+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:27.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:27 smithi151 bash[22565]: cluster 2024-09-10T18:50:26.161796+0000 mgr.y (mgr.24512) 622 : cluster [DBG] pgmap v1041: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:27.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:27 smithi151 bash[22565]: audit 2024-09-10T18:50:26.421081+0000 mon.a (mon.0) 3226 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:28.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:28 smithi151 bash[22565]: audit 2024-09-10T18:50:27.281869+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:28.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:28 smithi151 bash[22565]: cluster 2024-09-10T18:50:27.282033+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-09-10T18:50:28.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:28 smithi151 bash[22565]: audit 2024-09-10T18:50:27.283532+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130"}]: dispatch 2024-09-10T18:50:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:28 smithi100 bash[18597]: audit 2024-09-10T18:50:27.281869+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:28 smithi100 bash[18597]: cluster 2024-09-10T18:50:27.282033+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-09-10T18:50:28.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:28 smithi100 bash[18597]: audit 2024-09-10T18:50:27.283532+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130"}]: dispatch 2024-09-10T18:50:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:28 smithi100 bash[31607]: audit 2024-09-10T18:50:27.281869+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:28 smithi100 bash[31607]: cluster 2024-09-10T18:50:27.282033+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-09-10T18:50:28.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:28 smithi100 bash[31607]: audit 2024-09-10T18:50:27.283532+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130"}]: dispatch 2024-09-10T18:50:29.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:29 smithi151 bash[22565]: cluster 2024-09-10T18:50:28.162719+0000 mgr.y (mgr.24512) 623 : cluster [DBG] pgmap v1044: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:29.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:29 smithi151 bash[22565]: cluster 2024-09-10T18:50:28.276678+0000 mon.a (mon.0) 3230 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:29.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:29 smithi151 bash[22565]: audit 2024-09-10T18:50:28.284918+0000 mon.a (mon.0) 3231 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130"}]': finished 2024-09-10T18:50:29.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:29 smithi151 bash[22565]: cluster 2024-09-10T18:50:28.285063+0000 mon.a (mon.0) 3232 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-09-10T18:50:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[18597]: cluster 2024-09-10T18:50:28.162719+0000 mgr.y (mgr.24512) 623 : cluster [DBG] pgmap v1044: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[18597]: cluster 2024-09-10T18:50:28.276678+0000 mon.a (mon.0) 3230 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[18597]: audit 2024-09-10T18:50:28.284918+0000 mon.a (mon.0) 3231 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130"}]': finished 2024-09-10T18:50:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[18597]: cluster 2024-09-10T18:50:28.285063+0000 mon.a (mon.0) 3232 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-09-10T18:50:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[31607]: cluster 2024-09-10T18:50:28.162719+0000 mgr.y (mgr.24512) 623 : cluster [DBG] pgmap v1044: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[31607]: cluster 2024-09-10T18:50:28.276678+0000 mon.a (mon.0) 3230 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[31607]: audit 2024-09-10T18:50:28.284918+0000 mon.a (mon.0) 3231 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-130"}]': finished 2024-09-10T18:50:29.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:29 smithi100 bash[31607]: cluster 2024-09-10T18:50:28.285063+0000 mon.a (mon.0) 3232 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-09-10T18:50:30.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:30 smithi151 bash[22565]: cluster 2024-09-10T18:50:29.310264+0000 mon.a (mon.0) 3233 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-09-10T18:50:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:30 smithi100 bash[18597]: cluster 2024-09-10T18:50:29.310264+0000 mon.a (mon.0) 3233 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-09-10T18:50:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:30 smithi100 bash[31607]: cluster 2024-09-10T18:50:29.310264+0000 mon.a (mon.0) 3233 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-09-10T18:50:31.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:31 smithi151 bash[22565]: cluster 2024-09-10T18:50:30.163652+0000 mgr.y (mgr.24512) 624 : cluster [DBG] pgmap v1047: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:31.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:31 smithi151 bash[22565]: cluster 2024-09-10T18:50:30.323518+0000 mon.a (mon.0) 3234 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-09-10T18:50:31.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:31 smithi151 bash[22565]: audit 2024-09-10T18:50:30.327425+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:31 smithi100 bash[18597]: cluster 2024-09-10T18:50:30.163652+0000 mgr.y (mgr.24512) 624 : cluster [DBG] pgmap v1047: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:31 smithi100 bash[18597]: cluster 2024-09-10T18:50:30.323518+0000 mon.a (mon.0) 3234 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-09-10T18:50:31.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:31 smithi100 bash[18597]: audit 2024-09-10T18:50:30.327425+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:31.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:50:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:31] "GET /metrics HTTP/1.1" 200 228691 "" "Prometheus/2.43.0" 2024-09-10T18:50:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:31 smithi100 bash[31607]: cluster 2024-09-10T18:50:30.163652+0000 mgr.y (mgr.24512) 624 : cluster [DBG] pgmap v1047: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:31 smithi100 bash[31607]: cluster 2024-09-10T18:50:30.323518+0000 mon.a (mon.0) 3234 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-09-10T18:50:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:31 smithi100 bash[31607]: audit 2024-09-10T18:50:30.327425+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:32.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:32 smithi151 bash[22565]: audit 2024-09-10T18:50:31.335051+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:32.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:32 smithi151 bash[22565]: cluster 2024-09-10T18:50:31.335509+0000 mon.a (mon.0) 3237 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-09-10T18:50:32.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:32 smithi151 bash[22565]: audit 2024-09-10T18:50:31.339980+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:32.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:32 smithi100 bash[18597]: audit 2024-09-10T18:50:31.335051+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:32.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:32 smithi100 bash[18597]: cluster 2024-09-10T18:50:31.335509+0000 mon.a (mon.0) 3237 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-09-10T18:50:32.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:32 smithi100 bash[18597]: audit 2024-09-10T18:50:31.339980+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:32.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:32 smithi100 bash[31607]: audit 2024-09-10T18:50:31.335051+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:32.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:32 smithi100 bash[31607]: cluster 2024-09-10T18:50:31.335509+0000 mon.a (mon.0) 3237 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-09-10T18:50:32.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:32 smithi100 bash[31607]: audit 2024-09-10T18:50:31.339980+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:33.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:50:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:50:33.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: cluster 2024-09-10T18:50:32.164606+0000 mgr.y (mgr.24512) 625 : cluster [DBG] pgmap v1050: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:33.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: audit 2024-09-10T18:50:32.339171+0000 mon.a (mon.0) 3239 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: cluster 2024-09-10T18:50:32.339347+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-09-10T18:50:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: audit 2024-09-10T18:50:32.340519+0000 mon.a (mon.0) 3241 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: audit 2024-09-10T18:50:32.791609+0000 mgr.y (mgr.24512) 626 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: audit 2024-09-10T18:50:33.338914+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: cluster 2024-09-10T18:50:33.339017+0000 mon.a (mon.0) 3243 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-09-10T18:50:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:33 smithi151 bash[22565]: audit 2024-09-10T18:50:33.342419+0000 mon.a (mon.0) 3244 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-132", "mode": "writeback"}]: dispatch 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: cluster 2024-09-10T18:50:32.164606+0000 mgr.y (mgr.24512) 625 : cluster [DBG] pgmap v1050: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: audit 2024-09-10T18:50:32.339171+0000 mon.a (mon.0) 3239 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: cluster 2024-09-10T18:50:32.339347+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: audit 2024-09-10T18:50:32.340519+0000 mon.a (mon.0) 3241 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: audit 2024-09-10T18:50:32.791609+0000 mgr.y (mgr.24512) 626 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: audit 2024-09-10T18:50:33.338914+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: cluster 2024-09-10T18:50:33.339017+0000 mon.a (mon.0) 3243 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-09-10T18:50:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[18597]: audit 2024-09-10T18:50:33.342419+0000 mon.a (mon.0) 3244 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-132", "mode": "writeback"}]: dispatch 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: cluster 2024-09-10T18:50:32.164606+0000 mgr.y (mgr.24512) 625 : cluster [DBG] pgmap v1050: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: audit 2024-09-10T18:50:32.339171+0000 mon.a (mon.0) 3239 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: cluster 2024-09-10T18:50:32.339347+0000 mon.a (mon.0) 3240 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: audit 2024-09-10T18:50:32.340519+0000 mon.a (mon.0) 3241 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: audit 2024-09-10T18:50:32.791609+0000 mgr.y (mgr.24512) 626 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: audit 2024-09-10T18:50:33.338914+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: cluster 2024-09-10T18:50:33.339017+0000 mon.a (mon.0) 3243 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-09-10T18:50:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:33 smithi100 bash[31607]: audit 2024-09-10T18:50:33.342419+0000 mon.a (mon.0) 3244 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-132", "mode": "writeback"}]: dispatch 2024-09-10T18:50:34.671 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:50:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:34.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:34 smithi151 bash[22565]: cluster 2024-09-10T18:50:34.335526+0000 mon.a (mon.0) 3245 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:34.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:34 smithi151 bash[22565]: audit 2024-09-10T18:50:34.344912+0000 mon.a (mon.0) 3246 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-132", "mode": "writeback"}]': finished 2024-09-10T18:50:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:34 smithi100 bash[18597]: cluster 2024-09-10T18:50:34.335526+0000 mon.a (mon.0) 3245 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:34 smithi100 bash[18597]: audit 2024-09-10T18:50:34.344912+0000 mon.a (mon.0) 3246 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-132", "mode": "writeback"}]': finished 2024-09-10T18:50:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:34 smithi100 bash[31607]: cluster 2024-09-10T18:50:34.335526+0000 mon.a (mon.0) 3245 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:34 smithi100 bash[31607]: audit 2024-09-10T18:50:34.344912+0000 mon.a (mon.0) 3246 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-132", "mode": "writeback"}]': finished 2024-09-10T18:50:35.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:35 smithi151 bash[22565]: cluster 2024-09-10T18:50:34.165507+0000 mgr.y (mgr.24512) 627 : cluster [DBG] pgmap v1053: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:35.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:35 smithi151 bash[22565]: cluster 2024-09-10T18:50:34.345061+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-09-10T18:50:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:35 smithi100 bash[18597]: cluster 2024-09-10T18:50:34.165507+0000 mgr.y (mgr.24512) 627 : cluster [DBG] pgmap v1053: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:35 smithi100 bash[18597]: cluster 2024-09-10T18:50:34.345061+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-09-10T18:50:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:35 smithi100 bash[31607]: cluster 2024-09-10T18:50:34.165507+0000 mgr.y (mgr.24512) 627 : cluster [DBG] pgmap v1053: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:35 smithi100 bash[31607]: cluster 2024-09-10T18:50:34.345061+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-09-10T18:50:36.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:36 smithi151 bash[22565]: cluster 2024-09-10T18:50:35.368007+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-09-10T18:50:36.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:36 smithi100 bash[31607]: cluster 2024-09-10T18:50:35.368007+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-09-10T18:50:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:36 smithi100 bash[18597]: cluster 2024-09-10T18:50:35.368007+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-09-10T18:50:37.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:37 smithi151 bash[22565]: cluster 2024-09-10T18:50:36.166470+0000 mgr.y (mgr.24512) 628 : cluster [DBG] pgmap v1056: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:37.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:37 smithi151 bash[22565]: cluster 2024-09-10T18:50:36.371655+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-09-10T18:50:37.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:37 smithi151 bash[22565]: audit 2024-09-10T18:50:36.422678+0000 mon.a (mon.0) 3250 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:37.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:37 smithi100 bash[31607]: cluster 2024-09-10T18:50:36.166470+0000 mgr.y (mgr.24512) 628 : cluster [DBG] pgmap v1056: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:37 smithi100 bash[31607]: cluster 2024-09-10T18:50:36.371655+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-09-10T18:50:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:37 smithi100 bash[31607]: audit 2024-09-10T18:50:36.422678+0000 mon.a (mon.0) 3250 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:37 smithi100 bash[18597]: cluster 2024-09-10T18:50:36.166470+0000 mgr.y (mgr.24512) 628 : cluster [DBG] pgmap v1056: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:37 smithi100 bash[18597]: cluster 2024-09-10T18:50:36.371655+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-09-10T18:50:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:37 smithi100 bash[18597]: audit 2024-09-10T18:50:36.422678+0000 mon.a (mon.0) 3250 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:38.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:38 smithi151 bash[22565]: audit 2024-09-10T18:50:37.397170+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:38.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:38 smithi151 bash[22565]: cluster 2024-09-10T18:50:37.397257+0000 mon.a (mon.0) 3252 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-09-10T18:50:38.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:38 smithi151 bash[22565]: audit 2024-09-10T18:50:37.407016+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:38 smithi100 bash[18597]: audit 2024-09-10T18:50:37.397170+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:38 smithi100 bash[18597]: cluster 2024-09-10T18:50:37.397257+0000 mon.a (mon.0) 3252 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-09-10T18:50:38.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:38 smithi100 bash[18597]: audit 2024-09-10T18:50:37.407016+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:38 smithi100 bash[31607]: audit 2024-09-10T18:50:37.397170+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:38 smithi100 bash[31607]: cluster 2024-09-10T18:50:37.397257+0000 mon.a (mon.0) 3252 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-09-10T18:50:38.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:38 smithi100 bash[31607]: audit 2024-09-10T18:50:37.407016+0000 mon.a (mon.0) 3253 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:39.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:39 smithi151 bash[22565]: cluster 2024-09-10T18:50:38.167322+0000 mgr.y (mgr.24512) 629 : cluster [DBG] pgmap v1059: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:39 smithi151 bash[22565]: audit 2024-09-10T18:50:38.402300+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:39 smithi151 bash[22565]: cluster 2024-09-10T18:50:38.402611+0000 mon.a (mon.0) 3255 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-09-10T18:50:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:39 smithi151 bash[22565]: audit 2024-09-10T18:50:39.347731+0000 mon.a (mon.0) 3256 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:39 smithi151 bash[22565]: audit 2024-09-10T18:50:39.352728+0000 mon.c (mon.2) 513 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:39 smithi151 bash[22565]: cluster 2024-09-10T18:50:39.401894+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[18597]: cluster 2024-09-10T18:50:38.167322+0000 mgr.y (mgr.24512) 629 : cluster [DBG] pgmap v1059: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[18597]: audit 2024-09-10T18:50:38.402300+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[18597]: cluster 2024-09-10T18:50:38.402611+0000 mon.a (mon.0) 3255 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[18597]: audit 2024-09-10T18:50:39.347731+0000 mon.a (mon.0) 3256 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[18597]: audit 2024-09-10T18:50:39.352728+0000 mon.c (mon.2) 513 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[18597]: cluster 2024-09-10T18:50:39.401894+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[31607]: cluster 2024-09-10T18:50:38.167322+0000 mgr.y (mgr.24512) 629 : cluster [DBG] pgmap v1059: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[31607]: audit 2024-09-10T18:50:38.402300+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[31607]: cluster 2024-09-10T18:50:38.402611+0000 mon.a (mon.0) 3255 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-09-10T18:50:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[31607]: audit 2024-09-10T18:50:39.347731+0000 mon.a (mon.0) 3256 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[31607]: audit 2024-09-10T18:50:39.352728+0000 mon.c (mon.2) 513 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:39 smithi100 bash[31607]: cluster 2024-09-10T18:50:39.401894+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-09-10T18:50:40.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:40 smithi151 bash[22565]: audit 2024-09-10T18:50:39.433348+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:40.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:40 smithi100 bash[18597]: audit 2024-09-10T18:50:39.433348+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:40 smithi100 bash[31607]: audit 2024-09-10T18:50:39.433348+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:41.796 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:41] "GET /metrics HTTP/1.1" 200 228726 "" "Prometheus/2.43.0" 2024-09-10T18:50:41.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[18597]: cluster 2024-09-10T18:50:40.168120+0000 mgr.y (mgr.24512) 630 : cluster [DBG] pgmap v1062: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:41.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[18597]: audit 2024-09-10T18:50:40.424366+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:41.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[18597]: cluster 2024-09-10T18:50:40.424470+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-09-10T18:50:41.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[18597]: audit 2024-09-10T18:50:40.425521+0000 mon.a (mon.0) 3261 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:41.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[31607]: cluster 2024-09-10T18:50:40.168120+0000 mgr.y (mgr.24512) 630 : cluster [DBG] pgmap v1062: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:41.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[31607]: audit 2024-09-10T18:50:40.424366+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:41.797 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[31607]: cluster 2024-09-10T18:50:40.424470+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-09-10T18:50:41.798 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:41 smithi100 bash[31607]: audit 2024-09-10T18:50:40.425521+0000 mon.a (mon.0) 3261 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:41 smithi151 bash[22565]: cluster 2024-09-10T18:50:40.168120+0000 mgr.y (mgr.24512) 630 : cluster [DBG] pgmap v1062: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:41 smithi151 bash[22565]: audit 2024-09-10T18:50:40.424366+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:41 smithi151 bash[22565]: cluster 2024-09-10T18:50:40.424470+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-09-10T18:50:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:41 smithi151 bash[22565]: audit 2024-09-10T18:50:40.425521+0000 mon.a (mon.0) 3261 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132"}]: dispatch 2024-09-10T18:50:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:42 smithi100 bash[18597]: cluster 2024-09-10T18:50:41.419983+0000 mon.a (mon.0) 3262 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:42 smithi100 bash[18597]: audit 2024-09-10T18:50:41.435427+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:42 smithi100 bash[18597]: cluster 2024-09-10T18:50:41.435525+0000 mon.a (mon.0) 3264 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-09-10T18:50:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:42 smithi100 bash[31607]: cluster 2024-09-10T18:50:41.419983+0000 mon.a (mon.0) 3262 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:42 smithi100 bash[31607]: audit 2024-09-10T18:50:41.435427+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:42 smithi100 bash[31607]: cluster 2024-09-10T18:50:41.435525+0000 mon.a (mon.0) 3264 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-09-10T18:50:42.803 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:42 smithi151 bash[22565]: cluster 2024-09-10T18:50:41.419983+0000 mon.a (mon.0) 3262 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:42.803 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:42 smithi151 bash[22565]: audit 2024-09-10T18:50:41.435427+0000 mon.a (mon.0) 3263 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-132"}]': finished 2024-09-10T18:50:42.804 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:42 smithi151 bash[22565]: cluster 2024-09-10T18:50:41.435525+0000 mon.a (mon.0) 3264 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-09-10T18:50:43.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:50:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:50:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[18597]: cluster 2024-09-10T18:50:42.168974+0000 mgr.y (mgr.24512) 631 : cluster [DBG] pgmap v1065: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:50:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[18597]: cluster 2024-09-10T18:50:42.429812+0000 mon.a (mon.0) 3265 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[18597]: cluster 2024-09-10T18:50:42.442920+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-09-10T18:50:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[18597]: audit 2024-09-10T18:50:42.803042+0000 mgr.y (mgr.24512) 632 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[31607]: cluster 2024-09-10T18:50:42.168974+0000 mgr.y (mgr.24512) 631 : cluster [DBG] pgmap v1065: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:50:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[31607]: cluster 2024-09-10T18:50:42.429812+0000 mon.a (mon.0) 3265 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[31607]: cluster 2024-09-10T18:50:42.442920+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-09-10T18:50:43.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:43 smithi100 bash[31607]: audit 2024-09-10T18:50:42.803042+0000 mgr.y (mgr.24512) 632 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:43.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:43 smithi151 bash[22565]: cluster 2024-09-10T18:50:42.168974+0000 mgr.y (mgr.24512) 631 : cluster [DBG] pgmap v1065: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:50:43.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:43 smithi151 bash[22565]: cluster 2024-09-10T18:50:42.429812+0000 mon.a (mon.0) 3265 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:43.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:43 smithi151 bash[22565]: cluster 2024-09-10T18:50:42.442920+0000 mon.a (mon.0) 3266 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-09-10T18:50:43.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:43 smithi151 bash[22565]: audit 2024-09-10T18:50:42.803042+0000 mgr.y (mgr.24512) 632 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:44 smithi100 bash[18597]: cluster 2024-09-10T18:50:43.445749+0000 mon.a (mon.0) 3267 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-09-10T18:50:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:44 smithi100 bash[18597]: audit 2024-09-10T18:50:43.452752+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:44 smithi100 bash[31607]: cluster 2024-09-10T18:50:43.445749+0000 mon.a (mon.0) 3267 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-09-10T18:50:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:44 smithi100 bash[31607]: audit 2024-09-10T18:50:43.452752+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:44.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:50:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:44.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:44 smithi151 bash[22565]: cluster 2024-09-10T18:50:43.445749+0000 mon.a (mon.0) 3267 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-09-10T18:50:44.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:44 smithi151 bash[22565]: audit 2024-09-10T18:50:43.452752+0000 mon.a (mon.0) 3268 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[18597]: cluster 2024-09-10T18:50:44.169817+0000 mgr.y (mgr.24512) 633 : cluster [DBG] pgmap v1068: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:50:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[18597]: audit 2024-09-10T18:50:44.458475+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[18597]: cluster 2024-09-10T18:50:44.459460+0000 mon.a (mon.0) 3270 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-09-10T18:50:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[18597]: audit 2024-09-10T18:50:44.462896+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[31607]: cluster 2024-09-10T18:50:44.169817+0000 mgr.y (mgr.24512) 633 : cluster [DBG] pgmap v1068: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:50:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[31607]: audit 2024-09-10T18:50:44.458475+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[31607]: cluster 2024-09-10T18:50:44.459460+0000 mon.a (mon.0) 3270 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-09-10T18:50:45.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:45 smithi100 bash[31607]: audit 2024-09-10T18:50:44.462896+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:45.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:45 smithi151 bash[22565]: cluster 2024-09-10T18:50:44.169817+0000 mgr.y (mgr.24512) 633 : cluster [DBG] pgmap v1068: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:50:45.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:45 smithi151 bash[22565]: audit 2024-09-10T18:50:44.458475+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:45.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:45 smithi151 bash[22565]: cluster 2024-09-10T18:50:44.459460+0000 mon.a (mon.0) 3270 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-09-10T18:50:45.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:45 smithi151 bash[22565]: audit 2024-09-10T18:50:44.462896+0000 mon.a (mon.0) 3271 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[18597]: audit 2024-09-10T18:50:45.460279+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[18597]: cluster 2024-09-10T18:50:45.460350+0000 mon.a (mon.0) 3273 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-09-10T18:50:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[18597]: audit 2024-09-10T18:50:45.462182+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-134"}]: dispatch 2024-09-10T18:50:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[18597]: audit 2024-09-10T18:50:46.134436+0000 mon.a (mon.0) 3275 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-134"}]': finished 2024-09-10T18:50:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[18597]: cluster 2024-09-10T18:50:46.134882+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-09-10T18:50:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[18597]: audit 2024-09-10T18:50:46.137065+0000 mon.a (mon.0) 3277 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-134", "mode": "writeback"}]: dispatch 2024-09-10T18:50:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[31607]: audit 2024-09-10T18:50:45.460279+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[31607]: cluster 2024-09-10T18:50:45.460350+0000 mon.a (mon.0) 3273 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-09-10T18:50:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[31607]: audit 2024-09-10T18:50:45.462182+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-134"}]: dispatch 2024-09-10T18:50:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[31607]: audit 2024-09-10T18:50:46.134436+0000 mon.a (mon.0) 3275 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-134"}]': finished 2024-09-10T18:50:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[31607]: cluster 2024-09-10T18:50:46.134882+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-09-10T18:50:46.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:46 smithi100 bash[31607]: audit 2024-09-10T18:50:46.137065+0000 mon.a (mon.0) 3277 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-134", "mode": "writeback"}]: dispatch 2024-09-10T18:50:46.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:46 smithi151 bash[22565]: audit 2024-09-10T18:50:45.460279+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:46.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:46 smithi151 bash[22565]: cluster 2024-09-10T18:50:45.460350+0000 mon.a (mon.0) 3273 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-09-10T18:50:46.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:46 smithi151 bash[22565]: audit 2024-09-10T18:50:45.462182+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-134"}]: dispatch 2024-09-10T18:50:46.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:46 smithi151 bash[22565]: audit 2024-09-10T18:50:46.134436+0000 mon.a (mon.0) 3275 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-134"}]': finished 2024-09-10T18:50:46.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:46 smithi151 bash[22565]: cluster 2024-09-10T18:50:46.134882+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-09-10T18:50:46.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:46 smithi151 bash[22565]: audit 2024-09-10T18:50:46.137065+0000 mon.a (mon.0) 3277 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-134", "mode": "writeback"}]: dispatch 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[18597]: cluster 2024-09-10T18:50:46.170596+0000 mgr.y (mgr.24512) 634 : cluster [DBG] pgmap v1072: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[18597]: cluster 2024-09-10T18:50:47.128669+0000 mon.a (mon.0) 3278 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[18597]: audit 2024-09-10T18:50:47.142523+0000 mon.a (mon.0) 3279 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-134", "mode": "writeback"}]': finished 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[18597]: cluster 2024-09-10T18:50:47.142604+0000 mon.a (mon.0) 3280 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[18597]: audit 2024-09-10T18:50:47.208161+0000 mon.a (mon.0) 3281 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[31607]: cluster 2024-09-10T18:50:46.170596+0000 mgr.y (mgr.24512) 634 : cluster [DBG] pgmap v1072: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[31607]: cluster 2024-09-10T18:50:47.128669+0000 mon.a (mon.0) 3278 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[31607]: audit 2024-09-10T18:50:47.142523+0000 mon.a (mon.0) 3279 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-134", "mode": "writeback"}]': finished 2024-09-10T18:50:47.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[31607]: cluster 2024-09-10T18:50:47.142604+0000 mon.a (mon.0) 3280 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-09-10T18:50:47.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:47 smithi100 bash[31607]: audit 2024-09-10T18:50:47.208161+0000 mon.a (mon.0) 3281 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:47 smithi151 bash[22565]: cluster 2024-09-10T18:50:46.170596+0000 mgr.y (mgr.24512) 634 : cluster [DBG] pgmap v1072: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:47 smithi151 bash[22565]: cluster 2024-09-10T18:50:47.128669+0000 mon.a (mon.0) 3278 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:47 smithi151 bash[22565]: audit 2024-09-10T18:50:47.142523+0000 mon.a (mon.0) 3279 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-134", "mode": "writeback"}]': finished 2024-09-10T18:50:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:47 smithi151 bash[22565]: cluster 2024-09-10T18:50:47.142604+0000 mon.a (mon.0) 3280 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-09-10T18:50:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:47 smithi151 bash[22565]: audit 2024-09-10T18:50:47.208161+0000 mon.a (mon.0) 3281 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[18597]: audit 2024-09-10T18:50:47.827299+0000 mon.c (mon.2) 514 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[18597]: audit 2024-09-10T18:50:48.142436+0000 mon.a (mon.0) 3282 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[18597]: cluster 2024-09-10T18:50:48.142497+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-09-10T18:50:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[18597]: audit 2024-09-10T18:50:48.143371+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134"}]: dispatch 2024-09-10T18:50:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[18597]: audit 2024-09-10T18:50:48.277895+0000 mon.c (mon.2) 515 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[18597]: audit 2024-09-10T18:50:48.279688+0000 mon.c (mon.2) 516 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[18597]: audit 2024-09-10T18:50:48.290094+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[31607]: audit 2024-09-10T18:50:47.827299+0000 mon.c (mon.2) 514 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[31607]: audit 2024-09-10T18:50:48.142436+0000 mon.a (mon.0) 3282 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[31607]: cluster 2024-09-10T18:50:48.142497+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-09-10T18:50:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[31607]: audit 2024-09-10T18:50:48.143371+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134"}]: dispatch 2024-09-10T18:50:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[31607]: audit 2024-09-10T18:50:48.277895+0000 mon.c (mon.2) 515 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[31607]: audit 2024-09-10T18:50:48.279688+0000 mon.c (mon.2) 516 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:48.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:48 smithi100 bash[31607]: audit 2024-09-10T18:50:48.290094+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:48.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:48 smithi151 bash[22565]: audit 2024-09-10T18:50:47.827299+0000 mon.c (mon.2) 514 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:48.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:48 smithi151 bash[22565]: audit 2024-09-10T18:50:48.142436+0000 mon.a (mon.0) 3282 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:48.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:48 smithi151 bash[22565]: cluster 2024-09-10T18:50:48.142497+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-09-10T18:50:48.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:48 smithi151 bash[22565]: audit 2024-09-10T18:50:48.143371+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134"}]: dispatch 2024-09-10T18:50:48.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:48 smithi151 bash[22565]: audit 2024-09-10T18:50:48.277895+0000 mon.c (mon.2) 515 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:48.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:48 smithi151 bash[22565]: audit 2024-09-10T18:50:48.279688+0000 mon.c (mon.2) 516 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:48.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:48 smithi151 bash[22565]: audit 2024-09-10T18:50:48.290094+0000 mon.a (mon.0) 3285 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[18597]: cluster 2024-09-10T18:50:48.171077+0000 mgr.y (mgr.24512) 635 : cluster [DBG] pgmap v1075: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[18597]: cluster 2024-09-10T18:50:49.140064+0000 mon.a (mon.0) 3286 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[18597]: audit 2024-09-10T18:50:49.152489+0000 mon.a (mon.0) 3287 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134"}]': finished 2024-09-10T18:50:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[18597]: cluster 2024-09-10T18:50:49.152691+0000 mon.a (mon.0) 3288 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-09-10T18:50:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[31607]: cluster 2024-09-10T18:50:48.171077+0000 mgr.y (mgr.24512) 635 : cluster [DBG] pgmap v1075: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[31607]: cluster 2024-09-10T18:50:49.140064+0000 mon.a (mon.0) 3286 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[31607]: audit 2024-09-10T18:50:49.152489+0000 mon.a (mon.0) 3287 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134"}]': finished 2024-09-10T18:50:49.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:49 smithi100 bash[31607]: cluster 2024-09-10T18:50:49.152691+0000 mon.a (mon.0) 3288 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-09-10T18:50:49.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:49 smithi151 bash[22565]: cluster 2024-09-10T18:50:48.171077+0000 mgr.y (mgr.24512) 635 : cluster [DBG] pgmap v1075: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:50:49.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:49 smithi151 bash[22565]: cluster 2024-09-10T18:50:49.140064+0000 mon.a (mon.0) 3286 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:49.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:49 smithi151 bash[22565]: audit 2024-09-10T18:50:49.152489+0000 mon.a (mon.0) 3287 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-134"}]': finished 2024-09-10T18:50:49.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:49 smithi151 bash[22565]: cluster 2024-09-10T18:50:49.152691+0000 mon.a (mon.0) 3288 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-09-10T18:50:51.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:51 smithi151 bash[22565]: cluster 2024-09-10T18:50:50.157446+0000 mon.a (mon.0) 3289 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-09-10T18:50:51.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:51 smithi151 bash[22565]: cluster 2024-09-10T18:50:50.171505+0000 mgr.y (mgr.24512) 636 : cluster [DBG] pgmap v1078: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:51.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:51 smithi151 bash[22565]: cluster 2024-09-10T18:50:51.127045+0000 mon.a (mon.0) 3290 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:51.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:50:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:51] "GET /metrics HTTP/1.1" 200 228726 "" "Prometheus/2.43.0" 2024-09-10T18:50:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:51 smithi100 bash[18597]: cluster 2024-09-10T18:50:50.157446+0000 mon.a (mon.0) 3289 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-09-10T18:50:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:51 smithi100 bash[18597]: cluster 2024-09-10T18:50:50.171505+0000 mgr.y (mgr.24512) 636 : cluster [DBG] pgmap v1078: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:51.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:51 smithi100 bash[18597]: cluster 2024-09-10T18:50:51.127045+0000 mon.a (mon.0) 3290 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:51 smithi100 bash[31607]: cluster 2024-09-10T18:50:50.157446+0000 mon.a (mon.0) 3289 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-09-10T18:50:51.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:51 smithi100 bash[31607]: cluster 2024-09-10T18:50:50.171505+0000 mgr.y (mgr.24512) 636 : cluster [DBG] pgmap v1078: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:51.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:51 smithi100 bash[31607]: cluster 2024-09-10T18:50:51.127045+0000 mon.a (mon.0) 3290 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:52.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:52 smithi100 bash[18597]: cluster 2024-09-10T18:50:51.169113+0000 mon.a (mon.0) 3291 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-09-10T18:50:52.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:52 smithi100 bash[18597]: audit 2024-09-10T18:50:51.174454+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:52 smithi100 bash[31607]: cluster 2024-09-10T18:50:51.169113+0000 mon.a (mon.0) 3291 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-09-10T18:50:52.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:52 smithi100 bash[31607]: audit 2024-09-10T18:50:51.174454+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:52.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:52 smithi151 bash[22565]: cluster 2024-09-10T18:50:51.169113+0000 mon.a (mon.0) 3291 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-09-10T18:50:52.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:52 smithi151 bash[22565]: audit 2024-09-10T18:50:51.174454+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:53.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:50:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:50:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[18597]: audit 2024-09-10T18:50:52.174213+0000 mon.a (mon.0) 3293 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[18597]: cluster 2024-09-10T18:50:52.174634+0000 mon.a (mon.0) 3294 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-09-10T18:50:53.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[18597]: cluster 2024-09-10T18:50:52.180326+0000 mgr.y (mgr.24512) 637 : cluster [DBG] pgmap v1081: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:50:53.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[18597]: audit 2024-09-10T18:50:52.184451+0000 mon.a (mon.0) 3295 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:53.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[18597]: audit 2024-09-10T18:50:52.814505+0000 mgr.y (mgr.24512) 638 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[31607]: audit 2024-09-10T18:50:52.174213+0000 mon.a (mon.0) 3293 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[31607]: cluster 2024-09-10T18:50:52.174634+0000 mon.a (mon.0) 3294 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-09-10T18:50:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[31607]: cluster 2024-09-10T18:50:52.180326+0000 mgr.y (mgr.24512) 637 : cluster [DBG] pgmap v1081: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:50:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[31607]: audit 2024-09-10T18:50:52.184451+0000 mon.a (mon.0) 3295 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:53.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:53 smithi100 bash[31607]: audit 2024-09-10T18:50:52.814505+0000 mgr.y (mgr.24512) 638 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:53.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:53 smithi151 bash[22565]: audit 2024-09-10T18:50:52.174213+0000 mon.a (mon.0) 3293 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:53.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:53 smithi151 bash[22565]: cluster 2024-09-10T18:50:52.174634+0000 mon.a (mon.0) 3294 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-09-10T18:50:53.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:53 smithi151 bash[22565]: cluster 2024-09-10T18:50:52.180326+0000 mgr.y (mgr.24512) 637 : cluster [DBG] pgmap v1081: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:50:53.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:53 smithi151 bash[22565]: audit 2024-09-10T18:50:52.184451+0000 mon.a (mon.0) 3295 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:53.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:53 smithi151 bash[22565]: audit 2024-09-10T18:50:52.814505+0000 mgr.y (mgr.24512) 638 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:54 smithi100 bash[31607]: audit 2024-09-10T18:50:53.197831+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:54 smithi100 bash[31607]: cluster 2024-09-10T18:50:53.197980+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-09-10T18:50:54.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:54 smithi100 bash[31607]: audit 2024-09-10T18:50:53.198851+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-136"}]: dispatch 2024-09-10T18:50:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:54 smithi100 bash[18597]: audit 2024-09-10T18:50:53.197831+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:54 smithi100 bash[18597]: cluster 2024-09-10T18:50:53.197980+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-09-10T18:50:54.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:54 smithi100 bash[18597]: audit 2024-09-10T18:50:53.198851+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-136"}]: dispatch 2024-09-10T18:50:54.546 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:54 smithi151 bash[22565]: audit 2024-09-10T18:50:53.197831+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:54.547 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:54 smithi151 bash[22565]: cluster 2024-09-10T18:50:53.197980+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-09-10T18:50:54.547 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:54 smithi151 bash[22565]: audit 2024-09-10T18:50:53.198851+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-136"}]: dispatch 2024-09-10T18:50:54.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:50:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:50:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: cluster 2024-09-10T18:50:54.181103+0000 mgr.y (mgr.24512) 639 : cluster [DBG] pgmap v1083: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: audit 2024-09-10T18:50:54.203776+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-136"}]': finished 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: cluster 2024-09-10T18:50:54.204005+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: audit 2024-09-10T18:50:54.205096+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-136", "mode": "writeback"}]: dispatch 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: audit 2024-09-10T18:50:54.348097+0000 mon.a (mon.0) 3302 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: audit 2024-09-10T18:50:54.351017+0000 mon.c (mon.2) 517 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: cluster 2024-09-10T18:50:55.195728+0000 mon.a (mon.0) 3303 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:55.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: audit 2024-09-10T18:50:55.204983+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-136", "mode": "writeback"}]': finished 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[31607]: cluster 2024-09-10T18:50:55.205147+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: cluster 2024-09-10T18:50:54.181103+0000 mgr.y (mgr.24512) 639 : cluster [DBG] pgmap v1083: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: audit 2024-09-10T18:50:54.203776+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-136"}]': finished 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: cluster 2024-09-10T18:50:54.204005+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: audit 2024-09-10T18:50:54.205096+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-136", "mode": "writeback"}]: dispatch 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: audit 2024-09-10T18:50:54.348097+0000 mon.a (mon.0) 3302 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: audit 2024-09-10T18:50:54.351017+0000 mon.c (mon.2) 517 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: cluster 2024-09-10T18:50:55.195728+0000 mon.a (mon.0) 3303 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: audit 2024-09-10T18:50:55.204983+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-136", "mode": "writeback"}]': finished 2024-09-10T18:50:55.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:55 smithi100 bash[18597]: cluster 2024-09-10T18:50:55.205147+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-09-10T18:50:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: cluster 2024-09-10T18:50:54.181103+0000 mgr.y (mgr.24512) 639 : cluster [DBG] pgmap v1083: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:50:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: audit 2024-09-10T18:50:54.203776+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-136"}]': finished 2024-09-10T18:50:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: cluster 2024-09-10T18:50:54.204005+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-09-10T18:50:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: audit 2024-09-10T18:50:54.205096+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-136", "mode": "writeback"}]: dispatch 2024-09-10T18:50:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: audit 2024-09-10T18:50:54.348097+0000 mon.a (mon.0) 3302 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:50:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: audit 2024-09-10T18:50:54.351017+0000 mon.c (mon.2) 517 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: cluster 2024-09-10T18:50:55.195728+0000 mon.a (mon.0) 3303 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: audit 2024-09-10T18:50:55.204983+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-136", "mode": "writeback"}]': finished 2024-09-10T18:50:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:55 smithi151 bash[22565]: cluster 2024-09-10T18:50:55.205147+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-09-10T18:50:56.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:56 smithi100 bash[18597]: audit 2024-09-10T18:50:55.319421+0000 mon.a (mon.0) 3306 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:56.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:56 smithi100 bash[31607]: audit 2024-09-10T18:50:55.319421+0000 mon.a (mon.0) 3306 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:56.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:56 smithi151 bash[22565]: audit 2024-09-10T18:50:55.319421+0000 mon.a (mon.0) 3306 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[18597]: cluster 2024-09-10T18:50:56.183163+0000 mgr.y (mgr.24512) 640 : cluster [DBG] pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[18597]: audit 2024-09-10T18:50:56.226217+0000 mon.a (mon.0) 3307 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[18597]: cluster 2024-09-10T18:50:56.226297+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[18597]: audit 2024-09-10T18:50:56.227443+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136"}]: dispatch 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[31607]: cluster 2024-09-10T18:50:56.183163+0000 mgr.y (mgr.24512) 640 : cluster [DBG] pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[31607]: audit 2024-09-10T18:50:56.226217+0000 mon.a (mon.0) 3307 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[31607]: cluster 2024-09-10T18:50:56.226297+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-09-10T18:50:57.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:57 smithi100 bash[31607]: audit 2024-09-10T18:50:56.227443+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136"}]: dispatch 2024-09-10T18:50:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:57 smithi151 bash[22565]: cluster 2024-09-10T18:50:56.183163+0000 mgr.y (mgr.24512) 640 : cluster [DBG] pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:50:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:57 smithi151 bash[22565]: audit 2024-09-10T18:50:56.226217+0000 mon.a (mon.0) 3307 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:50:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:57 smithi151 bash[22565]: cluster 2024-09-10T18:50:56.226297+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-09-10T18:50:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:57 smithi151 bash[22565]: audit 2024-09-10T18:50:56.227443+0000 mon.a (mon.0) 3309 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136"}]: dispatch 2024-09-10T18:50:58.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:58 smithi100 bash[18597]: cluster 2024-09-10T18:50:57.220351+0000 mon.a (mon.0) 3310 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:58.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:58 smithi100 bash[18597]: audit 2024-09-10T18:50:57.230272+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136"}]': finished 2024-09-10T18:50:58.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:58 smithi100 bash[18597]: cluster 2024-09-10T18:50:57.230525+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-09-10T18:50:58.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:58 smithi100 bash[31607]: cluster 2024-09-10T18:50:57.220351+0000 mon.a (mon.0) 3310 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:58.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:58 smithi100 bash[31607]: audit 2024-09-10T18:50:57.230272+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136"}]': finished 2024-09-10T18:50:58.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:58 smithi100 bash[31607]: cluster 2024-09-10T18:50:57.230525+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-09-10T18:50:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:58 smithi151 bash[22565]: cluster 2024-09-10T18:50:57.220351+0000 mon.a (mon.0) 3310 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:58 smithi151 bash[22565]: audit 2024-09-10T18:50:57.230272+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-136"}]': finished 2024-09-10T18:50:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:58 smithi151 bash[22565]: cluster 2024-09-10T18:50:57.230525+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-09-10T18:50:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:59 smithi100 bash[18597]: cluster 2024-09-10T18:50:58.183989+0000 mgr.y (mgr.24512) 641 : cluster [DBG] pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:50:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:59 smithi100 bash[18597]: cluster 2024-09-10T18:50:58.235011+0000 mon.a (mon.0) 3313 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:50:59 smithi100 bash[18597]: cluster 2024-09-10T18:50:58.249066+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-09-10T18:50:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:59 smithi100 bash[31607]: cluster 2024-09-10T18:50:58.183989+0000 mgr.y (mgr.24512) 641 : cluster [DBG] pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:50:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:59 smithi100 bash[31607]: cluster 2024-09-10T18:50:58.235011+0000 mon.a (mon.0) 3313 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:59.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:50:59 smithi100 bash[31607]: cluster 2024-09-10T18:50:58.249066+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-09-10T18:50:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:59 smithi151 bash[22565]: cluster 2024-09-10T18:50:58.183989+0000 mgr.y (mgr.24512) 641 : cluster [DBG] pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:50:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:59 smithi151 bash[22565]: cluster 2024-09-10T18:50:58.235011+0000 mon.a (mon.0) 3313 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:59.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:50:59 smithi151 bash[22565]: cluster 2024-09-10T18:50:58.249066+0000 mon.a (mon.0) 3314 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-09-10T18:51:00.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:00 smithi100 bash[18597]: cluster 2024-09-10T18:50:59.252935+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-09-10T18:51:00.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:00 smithi100 bash[18597]: audit 2024-09-10T18:50:59.256317+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:00 smithi100 bash[31607]: cluster 2024-09-10T18:50:59.252935+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-09-10T18:51:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:00 smithi100 bash[31607]: audit 2024-09-10T18:50:59.256317+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:00 smithi151 bash[22565]: cluster 2024-09-10T18:50:59.252935+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-09-10T18:51:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:00 smithi151 bash[22565]: audit 2024-09-10T18:50:59.256317+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:01.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:01] "GET /metrics HTTP/1.1" 200 228735 "" "Prometheus/2.43.0" 2024-09-10T18:51:01.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[18597]: cluster 2024-09-10T18:51:00.184869+0000 mgr.y (mgr.24512) 642 : cluster [DBG] pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:01.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[18597]: audit 2024-09-10T18:51:00.258608+0000 mon.a (mon.0) 3317 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:01.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[18597]: cluster 2024-09-10T18:51:00.258862+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-09-10T18:51:01.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[18597]: audit 2024-09-10T18:51:00.263394+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:01.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[18597]: audit 2024-09-10T18:51:01.263890+0000 mon.a (mon.0) 3320 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:01.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[31607]: cluster 2024-09-10T18:51:00.184869+0000 mgr.y (mgr.24512) 642 : cluster [DBG] pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:01.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[31607]: audit 2024-09-10T18:51:00.258608+0000 mon.a (mon.0) 3317 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:01.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[31607]: cluster 2024-09-10T18:51:00.258862+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-09-10T18:51:01.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[31607]: audit 2024-09-10T18:51:00.263394+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:01.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:01 smithi100 bash[31607]: audit 2024-09-10T18:51:01.263890+0000 mon.a (mon.0) 3320 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:01 smithi151 bash[22565]: cluster 2024-09-10T18:51:00.184869+0000 mgr.y (mgr.24512) 642 : cluster [DBG] pgmap v1092: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:01 smithi151 bash[22565]: audit 2024-09-10T18:51:00.258608+0000 mon.a (mon.0) 3317 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:01 smithi151 bash[22565]: cluster 2024-09-10T18:51:00.258862+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-09-10T18:51:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:01 smithi151 bash[22565]: audit 2024-09-10T18:51:00.263394+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:01.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:01 smithi151 bash[22565]: audit 2024-09-10T18:51:01.263890+0000 mon.a (mon.0) 3320 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[18597]: cluster 2024-09-10T18:51:01.264103+0000 mon.a (mon.0) 3321 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[18597]: audit 2024-09-10T18:51:01.268003+0000 mon.a (mon.0) 3322 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[18597]: audit 2024-09-10T18:51:02.267474+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[18597]: cluster 2024-09-10T18:51:02.267700+0000 mon.a (mon.0) 3324 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[18597]: audit 2024-09-10T18:51:02.268802+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[31607]: cluster 2024-09-10T18:51:01.264103+0000 mon.a (mon.0) 3321 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[31607]: audit 2024-09-10T18:51:01.268003+0000 mon.a (mon.0) 3322 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:51:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[31607]: audit 2024-09-10T18:51:02.267474+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:51:02.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[31607]: cluster 2024-09-10T18:51:02.267700+0000 mon.a (mon.0) 3324 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-09-10T18:51:02.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:02 smithi100 bash[31607]: audit 2024-09-10T18:51:02.268802+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:51:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:02 smithi151 bash[22565]: cluster 2024-09-10T18:51:01.264103+0000 mon.a (mon.0) 3321 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-09-10T18:51:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:02 smithi151 bash[22565]: audit 2024-09-10T18:51:01.268003+0000 mon.a (mon.0) 3322 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:51:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:02 smithi151 bash[22565]: audit 2024-09-10T18:51:02.267474+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:51:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:02 smithi151 bash[22565]: cluster 2024-09-10T18:51:02.267700+0000 mon.a (mon.0) 3324 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-09-10T18:51:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:02 smithi151 bash[22565]: audit 2024-09-10T18:51:02.268802+0000 mon.a (mon.0) 3325 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:51:03.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:51:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:51:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[18597]: cluster 2024-09-10T18:51:02.185744+0000 mgr.y (mgr.24512) 643 : cluster [DBG] pgmap v1095: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[18597]: audit 2024-09-10T18:51:02.823586+0000 mgr.y (mgr.24512) 644 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[18597]: audit 2024-09-10T18:51:03.269444+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:51:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[18597]: cluster 2024-09-10T18:51:03.269551+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-09-10T18:51:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[18597]: audit 2024-09-10T18:51:03.271360+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:51:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[31607]: cluster 2024-09-10T18:51:02.185744+0000 mgr.y (mgr.24512) 643 : cluster [DBG] pgmap v1095: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[31607]: audit 2024-09-10T18:51:02.823586+0000 mgr.y (mgr.24512) 644 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[31607]: audit 2024-09-10T18:51:03.269444+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:51:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[31607]: cluster 2024-09-10T18:51:03.269551+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-09-10T18:51:03.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:03 smithi100 bash[31607]: audit 2024-09-10T18:51:03.271360+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:51:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:03 smithi151 bash[22565]: cluster 2024-09-10T18:51:02.185744+0000 mgr.y (mgr.24512) 643 : cluster [DBG] pgmap v1095: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:03 smithi151 bash[22565]: audit 2024-09-10T18:51:02.823586+0000 mgr.y (mgr.24512) 644 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:03 smithi151 bash[22565]: audit 2024-09-10T18:51:03.269444+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:51:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:03 smithi151 bash[22565]: cluster 2024-09-10T18:51:03.269551+0000 mon.a (mon.0) 3327 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-09-10T18:51:03.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:03 smithi151 bash[22565]: audit 2024-09-10T18:51:03.271360+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:51:04.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:51:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:05 smithi100 bash[18597]: cluster 2024-09-10T18:51:04.186563+0000 mgr.y (mgr.24512) 645 : cluster [DBG] pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:05 smithi100 bash[18597]: audit 2024-09-10T18:51:04.274033+0000 mon.a (mon.0) 3329 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:51:05.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:05 smithi100 bash[18597]: cluster 2024-09-10T18:51:04.274490+0000 mon.a (mon.0) 3330 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-09-10T18:51:05.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:05 smithi100 bash[31607]: cluster 2024-09-10T18:51:04.186563+0000 mgr.y (mgr.24512) 645 : cluster [DBG] pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:05.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:05 smithi100 bash[31607]: audit 2024-09-10T18:51:04.274033+0000 mon.a (mon.0) 3329 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:51:05.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:05 smithi100 bash[31607]: cluster 2024-09-10T18:51:04.274490+0000 mon.a (mon.0) 3330 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-09-10T18:51:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:05 smithi151 bash[22565]: cluster 2024-09-10T18:51:04.186563+0000 mgr.y (mgr.24512) 645 : cluster [DBG] pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:05 smithi151 bash[22565]: audit 2024-09-10T18:51:04.274033+0000 mon.a (mon.0) 3329 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:51:05.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:05 smithi151 bash[22565]: cluster 2024-09-10T18:51:04.274490+0000 mon.a (mon.0) 3330 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-09-10T18:51:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:06 smithi100 bash[18597]: audit 2024-09-10T18:51:05.305582+0000 mon.a (mon.0) 3331 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:06 smithi100 bash[18597]: audit 2024-09-10T18:51:05.306537+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138"}]: dispatch 2024-09-10T18:51:06.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:06 smithi100 bash[18597]: cluster 2024-09-10T18:51:06.131181+0000 mon.a (mon.0) 3333 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:06.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:06 smithi100 bash[31607]: audit 2024-09-10T18:51:05.305582+0000 mon.a (mon.0) 3331 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:06.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:06 smithi100 bash[31607]: audit 2024-09-10T18:51:05.306537+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138"}]: dispatch 2024-09-10T18:51:06.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:06 smithi100 bash[31607]: cluster 2024-09-10T18:51:06.131181+0000 mon.a (mon.0) 3333 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:06 smithi151 bash[22565]: audit 2024-09-10T18:51:05.305582+0000 mon.a (mon.0) 3331 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:06 smithi151 bash[22565]: audit 2024-09-10T18:51:05.306537+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138"}]: dispatch 2024-09-10T18:51:06.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:06 smithi151 bash[22565]: cluster 2024-09-10T18:51:06.131181+0000 mon.a (mon.0) 3333 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:08.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:07 smithi100 bash[18597]: cluster 2024-09-10T18:51:06.188615+0000 mgr.y (mgr.24512) 646 : cluster [DBG] pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:08.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:07 smithi100 bash[18597]: audit 2024-09-10T18:51:06.280712+0000 mon.a (mon.0) 3334 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138"}]': finished 2024-09-10T18:51:08.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:07 smithi100 bash[18597]: cluster 2024-09-10T18:51:06.280790+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-09-10T18:51:08.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:07 smithi100 bash[31607]: cluster 2024-09-10T18:51:06.188615+0000 mgr.y (mgr.24512) 646 : cluster [DBG] pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:08.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:07 smithi100 bash[31607]: audit 2024-09-10T18:51:06.280712+0000 mon.a (mon.0) 3334 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138"}]': finished 2024-09-10T18:51:08.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:07 smithi100 bash[31607]: cluster 2024-09-10T18:51:06.280790+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-09-10T18:51:08.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:07 smithi151 bash[22565]: cluster 2024-09-10T18:51:06.188615+0000 mgr.y (mgr.24512) 646 : cluster [DBG] pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:08.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:07 smithi151 bash[22565]: audit 2024-09-10T18:51:06.280712+0000 mon.a (mon.0) 3334 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-138"}]': finished 2024-09-10T18:51:08.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:07 smithi151 bash[22565]: cluster 2024-09-10T18:51:06.280790+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-09-10T18:51:09.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:08 smithi100 bash[18597]: cluster 2024-09-10T18:51:07.290846+0000 mon.a (mon.0) 3336 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-09-10T18:51:09.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:08 smithi100 bash[18597]: cluster 2024-09-10T18:51:08.287416+0000 mon.a (mon.0) 3337 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-09-10T18:51:09.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:08 smithi100 bash[18597]: audit 2024-09-10T18:51:08.297099+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:09.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:08 smithi100 bash[31607]: cluster 2024-09-10T18:51:07.290846+0000 mon.a (mon.0) 3336 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-09-10T18:51:09.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:08 smithi100 bash[31607]: cluster 2024-09-10T18:51:08.287416+0000 mon.a (mon.0) 3337 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-09-10T18:51:09.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:08 smithi100 bash[31607]: audit 2024-09-10T18:51:08.297099+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:09.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:08 smithi151 bash[22565]: cluster 2024-09-10T18:51:07.290846+0000 mon.a (mon.0) 3336 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-09-10T18:51:09.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:08 smithi151 bash[22565]: cluster 2024-09-10T18:51:08.287416+0000 mon.a (mon.0) 3337 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-09-10T18:51:09.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:08 smithi151 bash[22565]: audit 2024-09-10T18:51:08.297099+0000 mon.a (mon.0) 3338 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:10.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[18597]: cluster 2024-09-10T18:51:08.189463+0000 mgr.y (mgr.24512) 647 : cluster [DBG] pgmap v1103: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:10.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[18597]: audit 2024-09-10T18:51:09.291467+0000 mon.a (mon.0) 3339 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:10.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[18597]: cluster 2024-09-10T18:51:09.291925+0000 mon.a (mon.0) 3340 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-09-10T18:51:10.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[18597]: audit 2024-09-10T18:51:09.295571+0000 mon.a (mon.0) 3341 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:10.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[18597]: audit 2024-09-10T18:51:09.341458+0000 mon.c (mon.2) 518 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:10.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[31607]: cluster 2024-09-10T18:51:08.189463+0000 mgr.y (mgr.24512) 647 : cluster [DBG] pgmap v1103: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:10.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[31607]: audit 2024-09-10T18:51:09.291467+0000 mon.a (mon.0) 3339 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:10.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[31607]: cluster 2024-09-10T18:51:09.291925+0000 mon.a (mon.0) 3340 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-09-10T18:51:10.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[31607]: audit 2024-09-10T18:51:09.295571+0000 mon.a (mon.0) 3341 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:10.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:09 smithi100 bash[31607]: audit 2024-09-10T18:51:09.341458+0000 mon.c (mon.2) 518 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:10.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:09 smithi151 bash[22565]: cluster 2024-09-10T18:51:08.189463+0000 mgr.y (mgr.24512) 647 : cluster [DBG] pgmap v1103: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:10.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:09 smithi151 bash[22565]: audit 2024-09-10T18:51:09.291467+0000 mon.a (mon.0) 3339 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:10.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:09 smithi151 bash[22565]: cluster 2024-09-10T18:51:09.291925+0000 mon.a (mon.0) 3340 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-09-10T18:51:10.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:09 smithi151 bash[22565]: audit 2024-09-10T18:51:09.295571+0000 mon.a (mon.0) 3341 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:10.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:09 smithi151 bash[22565]: audit 2024-09-10T18:51:09.341458+0000 mon.c (mon.2) 518 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:11 smithi151 bash[22565]: cluster 2024-09-10T18:51:10.190315+0000 mgr.y (mgr.24512) 648 : cluster [DBG] pgmap v1106: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:11 smithi151 bash[22565]: audit 2024-09-10T18:51:10.293301+0000 mon.a (mon.0) 3342 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:11.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:11 smithi151 bash[22565]: cluster 2024-09-10T18:51:10.293429+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-09-10T18:51:11.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:11 smithi151 bash[22565]: audit 2024-09-10T18:51:10.297246+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:51:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:11] "GET /metrics HTTP/1.1" 200 228732 "" "Prometheus/2.43.0" 2024-09-10T18:51:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[18597]: cluster 2024-09-10T18:51:10.190315+0000 mgr.y (mgr.24512) 648 : cluster [DBG] pgmap v1106: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[18597]: audit 2024-09-10T18:51:10.293301+0000 mon.a (mon.0) 3342 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[18597]: cluster 2024-09-10T18:51:10.293429+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-09-10T18:51:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[18597]: audit 2024-09-10T18:51:10.297246+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:51:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[31607]: cluster 2024-09-10T18:51:10.190315+0000 mgr.y (mgr.24512) 648 : cluster [DBG] pgmap v1106: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[31607]: audit 2024-09-10T18:51:10.293301+0000 mon.a (mon.0) 3342 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[31607]: cluster 2024-09-10T18:51:10.293429+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-09-10T18:51:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:11 smithi100 bash[31607]: audit 2024-09-10T18:51:10.297246+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:51:12.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:12 smithi151 bash[22565]: audit 2024-09-10T18:51:11.296666+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:51:12.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:12 smithi151 bash[22565]: cluster 2024-09-10T18:51:11.296827+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-09-10T18:51:12.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:12 smithi151 bash[22565]: audit 2024-09-10T18:51:11.298615+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:51:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:12 smithi100 bash[18597]: audit 2024-09-10T18:51:11.296666+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:51:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:12 smithi100 bash[18597]: cluster 2024-09-10T18:51:11.296827+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-09-10T18:51:12.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:12 smithi100 bash[18597]: audit 2024-09-10T18:51:11.298615+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:51:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:12 smithi100 bash[31607]: audit 2024-09-10T18:51:11.296666+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:51:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:12 smithi100 bash[31607]: cluster 2024-09-10T18:51:11.296827+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-09-10T18:51:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:12 smithi100 bash[31607]: audit 2024-09-10T18:51:11.298615+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:51:13.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:51:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:51:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: cluster 2024-09-10T18:51:12.191159+0000 mgr.y (mgr.24512) 649 : cluster [DBG] pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: audit 2024-09-10T18:51:12.302453+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:51:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: cluster 2024-09-10T18:51:12.302651+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-09-10T18:51:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: audit 2024-09-10T18:51:12.305201+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:51:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: audit 2024-09-10T18:51:12.831598+0000 mgr.y (mgr.24512) 650 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: audit 2024-09-10T18:51:13.300924+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:51:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: cluster 2024-09-10T18:51:13.301143+0000 mon.a (mon.0) 3352 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-09-10T18:51:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:13 smithi151 bash[22565]: audit 2024-09-10T18:51:13.302298+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:51:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: cluster 2024-09-10T18:51:12.191159+0000 mgr.y (mgr.24512) 649 : cluster [DBG] pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: audit 2024-09-10T18:51:12.302453+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:51:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: cluster 2024-09-10T18:51:12.302651+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-09-10T18:51:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: audit 2024-09-10T18:51:12.305201+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:51:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: audit 2024-09-10T18:51:12.831598+0000 mgr.y (mgr.24512) 650 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: audit 2024-09-10T18:51:13.300924+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:51:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: cluster 2024-09-10T18:51:13.301143+0000 mon.a (mon.0) 3352 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[18597]: audit 2024-09-10T18:51:13.302298+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: cluster 2024-09-10T18:51:12.191159+0000 mgr.y (mgr.24512) 649 : cluster [DBG] pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: audit 2024-09-10T18:51:12.302453+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: cluster 2024-09-10T18:51:12.302651+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: audit 2024-09-10T18:51:12.305201+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: audit 2024-09-10T18:51:12.831598+0000 mgr.y (mgr.24512) 650 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: audit 2024-09-10T18:51:13.300924+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: cluster 2024-09-10T18:51:13.301143+0000 mon.a (mon.0) 3352 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-09-10T18:51:13.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:13 smithi100 bash[31607]: audit 2024-09-10T18:51:13.302298+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:51:14.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:51:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:15.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:15 smithi151 bash[22565]: cluster 2024-09-10T18:51:14.192010+0000 mgr.y (mgr.24512) 651 : cluster [DBG] pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:15.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:15 smithi151 bash[22565]: audit 2024-09-10T18:51:14.304732+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:51:15.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:15 smithi151 bash[22565]: cluster 2024-09-10T18:51:14.304883+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-09-10T18:51:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:15 smithi100 bash[31607]: cluster 2024-09-10T18:51:14.192010+0000 mgr.y (mgr.24512) 651 : cluster [DBG] pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:15 smithi100 bash[31607]: audit 2024-09-10T18:51:14.304732+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:51:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:15 smithi100 bash[31607]: cluster 2024-09-10T18:51:14.304883+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-09-10T18:51:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:15 smithi100 bash[18597]: cluster 2024-09-10T18:51:14.192010+0000 mgr.y (mgr.24512) 651 : cluster [DBG] pgmap v1112: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:15 smithi100 bash[18597]: audit 2024-09-10T18:51:14.304732+0000 mon.a (mon.0) 3354 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:51:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:15 smithi100 bash[18597]: cluster 2024-09-10T18:51:14.304883+0000 mon.a (mon.0) 3355 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-09-10T18:51:17.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:17 smithi151 bash[22565]: cluster 2024-09-10T18:51:16.194113+0000 mgr.y (mgr.24512) 652 : cluster [DBG] pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-10T18:51:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:17 smithi100 bash[18597]: cluster 2024-09-10T18:51:16.194113+0000 mgr.y (mgr.24512) 652 : cluster [DBG] pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-10T18:51:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:17 smithi100 bash[31607]: cluster 2024-09-10T18:51:16.194113+0000 mgr.y (mgr.24512) 652 : cluster [DBG] pgmap v1114: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-10T18:51:19.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:19 smithi151 bash[22565]: cluster 2024-09-10T18:51:18.194977+0000 mgr.y (mgr.24512) 653 : cluster [DBG] pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-10T18:51:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:19 smithi100 bash[18597]: cluster 2024-09-10T18:51:18.194977+0000 mgr.y (mgr.24512) 653 : cluster [DBG] pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-10T18:51:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:19 smithi100 bash[31607]: cluster 2024-09-10T18:51:18.194977+0000 mgr.y (mgr.24512) 653 : cluster [DBG] pgmap v1115: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-10T18:51:21.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:51:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:21] "GET /metrics HTTP/1.1" 200 228732 "" "Prometheus/2.43.0" 2024-09-10T18:51:22.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:22 smithi151 bash[22565]: cluster 2024-09-10T18:51:20.195826+0000 mgr.y (mgr.24512) 654 : cluster [DBG] pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-09-10T18:51:22.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:22 smithi100 bash[18597]: cluster 2024-09-10T18:51:20.195826+0000 mgr.y (mgr.24512) 654 : cluster [DBG] pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-09-10T18:51:22.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:22 smithi100 bash[31607]: cluster 2024-09-10T18:51:20.195826+0000 mgr.y (mgr.24512) 654 : cluster [DBG] pgmap v1116: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 648 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-09-10T18:51:23.151 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:51:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:51:23.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:23 smithi151 bash[22565]: cluster 2024-09-10T18:51:22.197993+0000 mgr.y (mgr.24512) 655 : cluster [DBG] pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.5 KiB/s wr, 2 op/s 2024-09-10T18:51:23.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:23 smithi151 bash[22565]: audit 2024-09-10T18:51:22.843013+0000 mgr.y (mgr.24512) 656 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:23.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:23 smithi100 bash[18597]: cluster 2024-09-10T18:51:22.197993+0000 mgr.y (mgr.24512) 655 : cluster [DBG] pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.5 KiB/s wr, 2 op/s 2024-09-10T18:51:23.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:23 smithi100 bash[18597]: audit 2024-09-10T18:51:22.843013+0000 mgr.y (mgr.24512) 656 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:23 smithi100 bash[31607]: cluster 2024-09-10T18:51:22.197993+0000 mgr.y (mgr.24512) 655 : cluster [DBG] pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.5 KiB/s wr, 2 op/s 2024-09-10T18:51:23.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:23 smithi100 bash[31607]: audit 2024-09-10T18:51:22.843013+0000 mgr.y (mgr.24512) 656 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:24.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:51:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:25.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:25 smithi100 bash[31607]: cluster 2024-09-10T18:51:24.198795+0000 mgr.y (mgr.24512) 657 : cluster [DBG] pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-09-10T18:51:25.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:25 smithi100 bash[31607]: audit 2024-09-10T18:51:24.342013+0000 mon.c (mon.2) 519 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:25 smithi100 bash[18597]: cluster 2024-09-10T18:51:24.198795+0000 mgr.y (mgr.24512) 657 : cluster [DBG] pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-09-10T18:51:25.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:25 smithi100 bash[18597]: audit 2024-09-10T18:51:24.342013+0000 mon.c (mon.2) 519 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:25.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:25 smithi151 bash[22565]: cluster 2024-09-10T18:51:24.198795+0000 mgr.y (mgr.24512) 657 : cluster [DBG] pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-09-10T18:51:25.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:25 smithi151 bash[22565]: audit 2024-09-10T18:51:24.342013+0000 mon.c (mon.2) 519 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:26.671 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:51:26 smithi151 bash[57256]: logger=sqlstore.transactions t=2024-09-10T18:51:26.25848416Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:51:27.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:27 smithi100 bash[31607]: cluster 2024-09-10T18:51:26.200892+0000 mgr.y (mgr.24512) 658 : cluster [DBG] pgmap v1119: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:51:27.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:27 smithi100 bash[31607]: audit 2024-09-10T18:51:26.388680+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:27.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:27 smithi100 bash[31607]: audit 2024-09-10T18:51:26.389517+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140"}]: dispatch 2024-09-10T18:51:27.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:27 smithi100 bash[18597]: cluster 2024-09-10T18:51:26.200892+0000 mgr.y (mgr.24512) 658 : cluster [DBG] pgmap v1119: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:51:27.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:27 smithi100 bash[18597]: audit 2024-09-10T18:51:26.388680+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:27.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:27 smithi100 bash[18597]: audit 2024-09-10T18:51:26.389517+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140"}]: dispatch 2024-09-10T18:51:27.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:27 smithi151 bash[22565]: cluster 2024-09-10T18:51:26.200892+0000 mgr.y (mgr.24512) 658 : cluster [DBG] pgmap v1119: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:51:27.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:27 smithi151 bash[22565]: audit 2024-09-10T18:51:26.388680+0000 mon.a (mon.0) 3356 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:27.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:27 smithi151 bash[22565]: audit 2024-09-10T18:51:26.389517+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140"}]: dispatch 2024-09-10T18:51:28.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:28 smithi100 bash[31607]: audit 2024-09-10T18:51:27.271634+0000 mon.a (mon.0) 3358 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140"}]': finished 2024-09-10T18:51:28.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:28 smithi100 bash[31607]: cluster 2024-09-10T18:51:27.271748+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-09-10T18:51:28.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:28 smithi100 bash[18597]: audit 2024-09-10T18:51:27.271634+0000 mon.a (mon.0) 3358 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140"}]': finished 2024-09-10T18:51:28.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:28 smithi100 bash[18597]: cluster 2024-09-10T18:51:27.271748+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-09-10T18:51:28.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:28 smithi151 bash[22565]: audit 2024-09-10T18:51:27.271634+0000 mon.a (mon.0) 3358 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-140"}]': finished 2024-09-10T18:51:28.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:28 smithi151 bash[22565]: cluster 2024-09-10T18:51:27.271748+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-09-10T18:51:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:29 smithi100 bash[18597]: cluster 2024-09-10T18:51:28.201739+0000 mgr.y (mgr.24512) 659 : cluster [DBG] pgmap v1121: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:51:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:29 smithi100 bash[18597]: cluster 2024-09-10T18:51:28.275279+0000 mon.a (mon.0) 3360 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:29.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:29 smithi100 bash[18597]: cluster 2024-09-10T18:51:28.295588+0000 mon.a (mon.0) 3361 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-09-10T18:51:29.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:29 smithi100 bash[31607]: cluster 2024-09-10T18:51:28.201739+0000 mgr.y (mgr.24512) 659 : cluster [DBG] pgmap v1121: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:51:29.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:29 smithi100 bash[31607]: cluster 2024-09-10T18:51:28.275279+0000 mon.a (mon.0) 3360 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:29.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:29 smithi100 bash[31607]: cluster 2024-09-10T18:51:28.295588+0000 mon.a (mon.0) 3361 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-09-10T18:51:29.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:29 smithi151 bash[22565]: cluster 2024-09-10T18:51:28.201739+0000 mgr.y (mgr.24512) 659 : cluster [DBG] pgmap v1121: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:51:29.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:29 smithi151 bash[22565]: cluster 2024-09-10T18:51:28.275279+0000 mon.a (mon.0) 3360 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:29.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:29 smithi151 bash[22565]: cluster 2024-09-10T18:51:28.295588+0000 mon.a (mon.0) 3361 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-09-10T18:51:30.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:30 smithi151 bash[22565]: cluster 2024-09-10T18:51:29.307664+0000 mon.a (mon.0) 3362 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-09-10T18:51:30.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:30 smithi151 bash[22565]: audit 2024-09-10T18:51:29.310637+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:30 smithi100 bash[31607]: cluster 2024-09-10T18:51:29.307664+0000 mon.a (mon.0) 3362 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-09-10T18:51:30.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:30 smithi100 bash[31607]: audit 2024-09-10T18:51:29.310637+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:30 smithi100 bash[18597]: cluster 2024-09-10T18:51:29.307664+0000 mon.a (mon.0) 3362 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-09-10T18:51:30.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:30 smithi100 bash[18597]: audit 2024-09-10T18:51:29.310637+0000 mon.a (mon.0) 3363 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:31.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:31 smithi151 bash[22565]: cluster 2024-09-10T18:51:30.202525+0000 mgr.y (mgr.24512) 660 : cluster [DBG] pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:51:31.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:31 smithi151 bash[22565]: audit 2024-09-10T18:51:30.305738+0000 mon.a (mon.0) 3364 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:31.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:31 smithi151 bash[22565]: cluster 2024-09-10T18:51:30.306457+0000 mon.a (mon.0) 3365 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-09-10T18:51:31.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:31 smithi151 bash[22565]: audit 2024-09-10T18:51:30.369167+0000 mon.a (mon.0) 3366 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:31.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:31 smithi151 bash[22565]: audit 2024-09-10T18:51:31.309191+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:31.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:31 smithi151 bash[22565]: cluster 2024-09-10T18:51:31.309391+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-09-10T18:51:31.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:31 smithi151 bash[22565]: audit 2024-09-10T18:51:31.311754+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:31.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:31] "GET /metrics HTTP/1.1" 200 228761 "" "Prometheus/2.43.0" 2024-09-10T18:51:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[31607]: cluster 2024-09-10T18:51:30.202525+0000 mgr.y (mgr.24512) 660 : cluster [DBG] pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:51:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[31607]: audit 2024-09-10T18:51:30.305738+0000 mon.a (mon.0) 3364 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[31607]: cluster 2024-09-10T18:51:30.306457+0000 mon.a (mon.0) 3365 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-09-10T18:51:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[31607]: audit 2024-09-10T18:51:30.369167+0000 mon.a (mon.0) 3366 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[31607]: audit 2024-09-10T18:51:31.309191+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[31607]: cluster 2024-09-10T18:51:31.309391+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[31607]: audit 2024-09-10T18:51:31.311754+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18597]: cluster 2024-09-10T18:51:30.202525+0000 mgr.y (mgr.24512) 660 : cluster [DBG] pgmap v1124: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18597]: audit 2024-09-10T18:51:30.305738+0000 mon.a (mon.0) 3364 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18597]: cluster 2024-09-10T18:51:30.306457+0000 mon.a (mon.0) 3365 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18597]: audit 2024-09-10T18:51:30.369167+0000 mon.a (mon.0) 3366 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18597]: audit 2024-09-10T18:51:31.309191+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18597]: cluster 2024-09-10T18:51:31.309391+0000 mon.a (mon.0) 3368 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-09-10T18:51:31.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:31 smithi100 bash[18597]: audit 2024-09-10T18:51:31.311754+0000 mon.a (mon.0) 3369 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:33.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:51:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:51:33.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:33 smithi151 bash[22565]: cluster 2024-09-10T18:51:32.203369+0000 mgr.y (mgr.24512) 661 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:51:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:33 smithi151 bash[22565]: audit 2024-09-10T18:51:32.310841+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-142"}]': finished 2024-09-10T18:51:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:33 smithi151 bash[22565]: cluster 2024-09-10T18:51:32.311019+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-09-10T18:51:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:33 smithi151 bash[22565]: audit 2024-09-10T18:51:32.312190+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-142", "mode": "writeback"}]: dispatch 2024-09-10T18:51:33.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:33 smithi151 bash[22565]: audit 2024-09-10T18:51:32.854486+0000 mgr.y (mgr.24512) 662 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[18597]: cluster 2024-09-10T18:51:32.203369+0000 mgr.y (mgr.24512) 661 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:51:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[18597]: audit 2024-09-10T18:51:32.310841+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-142"}]': finished 2024-09-10T18:51:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[18597]: cluster 2024-09-10T18:51:32.311019+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-09-10T18:51:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[18597]: audit 2024-09-10T18:51:32.312190+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-142", "mode": "writeback"}]: dispatch 2024-09-10T18:51:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[18597]: audit 2024-09-10T18:51:32.854486+0000 mgr.y (mgr.24512) 662 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[31607]: cluster 2024-09-10T18:51:32.203369+0000 mgr.y (mgr.24512) 661 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:51:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[31607]: audit 2024-09-10T18:51:32.310841+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-142"}]': finished 2024-09-10T18:51:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[31607]: cluster 2024-09-10T18:51:32.311019+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-09-10T18:51:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[31607]: audit 2024-09-10T18:51:32.312190+0000 mon.a (mon.0) 3372 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-142", "mode": "writeback"}]: dispatch 2024-09-10T18:51:33.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:33 smithi100 bash[31607]: audit 2024-09-10T18:51:32.854486+0000 mgr.y (mgr.24512) 662 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:34.671 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:51:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:34.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:34 smithi151 bash[22565]: cluster 2024-09-10T18:51:33.304943+0000 mon.a (mon.0) 3373 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:34.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:34 smithi151 bash[22565]: audit 2024-09-10T18:51:33.313596+0000 mon.a (mon.0) 3374 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-142", "mode": "writeback"}]': finished 2024-09-10T18:51:34.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:34 smithi151 bash[22565]: cluster 2024-09-10T18:51:33.313801+0000 mon.a (mon.0) 3375 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-09-10T18:51:34.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:34 smithi151 bash[22565]: audit 2024-09-10T18:51:33.315787+0000 mon.a (mon.0) 3376 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:51:34.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:34 smithi151 bash[22565]: audit 2024-09-10T18:51:34.318313+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:51:34.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:34 smithi151 bash[22565]: cluster 2024-09-10T18:51:34.318452+0000 mon.a (mon.0) 3378 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[31607]: cluster 2024-09-10T18:51:33.304943+0000 mon.a (mon.0) 3373 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[31607]: audit 2024-09-10T18:51:33.313596+0000 mon.a (mon.0) 3374 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-142", "mode": "writeback"}]': finished 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[31607]: cluster 2024-09-10T18:51:33.313801+0000 mon.a (mon.0) 3375 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[31607]: audit 2024-09-10T18:51:33.315787+0000 mon.a (mon.0) 3376 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[31607]: audit 2024-09-10T18:51:34.318313+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[31607]: cluster 2024-09-10T18:51:34.318452+0000 mon.a (mon.0) 3378 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[18597]: cluster 2024-09-10T18:51:33.304943+0000 mon.a (mon.0) 3373 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[18597]: audit 2024-09-10T18:51:33.313596+0000 mon.a (mon.0) 3374 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-142", "mode": "writeback"}]': finished 2024-09-10T18:51:34.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[18597]: cluster 2024-09-10T18:51:33.313801+0000 mon.a (mon.0) 3375 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-09-10T18:51:34.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[18597]: audit 2024-09-10T18:51:33.315787+0000 mon.a (mon.0) 3376 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:51:34.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[18597]: audit 2024-09-10T18:51:34.318313+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:51:34.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:34 smithi100 bash[18597]: cluster 2024-09-10T18:51:34.318452+0000 mon.a (mon.0) 3378 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-09-10T18:51:35.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:35 smithi151 bash[22565]: cluster 2024-09-10T18:51:34.204187+0000 mgr.y (mgr.24512) 663 : cluster [DBG] pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-09-10T18:51:35.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:35 smithi151 bash[22565]: audit 2024-09-10T18:51:34.319678+0000 mon.a (mon.0) 3379 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:51:35.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:35 smithi151 bash[22565]: audit 2024-09-10T18:51:35.319635+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:51:35.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:35 smithi151 bash[22565]: cluster 2024-09-10T18:51:35.319797+0000 mon.a (mon.0) 3381 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-09-10T18:51:35.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:35 smithi151 bash[22565]: audit 2024-09-10T18:51:35.321998+0000 mon.a (mon.0) 3382 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:51:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[18597]: cluster 2024-09-10T18:51:34.204187+0000 mgr.y (mgr.24512) 663 : cluster [DBG] pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[18597]: audit 2024-09-10T18:51:34.319678+0000 mon.a (mon.0) 3379 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[18597]: audit 2024-09-10T18:51:35.319635+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[18597]: cluster 2024-09-10T18:51:35.319797+0000 mon.a (mon.0) 3381 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[18597]: audit 2024-09-10T18:51:35.321998+0000 mon.a (mon.0) 3382 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[31607]: cluster 2024-09-10T18:51:34.204187+0000 mgr.y (mgr.24512) 663 : cluster [DBG] pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[31607]: audit 2024-09-10T18:51:34.319678+0000 mon.a (mon.0) 3379 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[31607]: audit 2024-09-10T18:51:35.319635+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[31607]: cluster 2024-09-10T18:51:35.319797+0000 mon.a (mon.0) 3381 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-09-10T18:51:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:35 smithi100 bash[31607]: audit 2024-09-10T18:51:35.321998+0000 mon.a (mon.0) 3382 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:51:36.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:36 smithi151 bash[22565]: cluster 2024-09-10T18:51:36.140320+0000 mon.a (mon.0) 3383 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:36 smithi151 bash[22565]: cluster 2024-09-10T18:51:36.313201+0000 mon.a (mon.0) 3384 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:36 smithi151 bash[22565]: audit 2024-09-10T18:51:36.323962+0000 mon.a (mon.0) 3385 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:51:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:36 smithi151 bash[22565]: cluster 2024-09-10T18:51:36.324113+0000 mon.a (mon.0) 3386 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-09-10T18:51:36.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:36 smithi151 bash[22565]: audit 2024-09-10T18:51:36.325078+0000 mon.a (mon.0) 3387 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[18597]: cluster 2024-09-10T18:51:36.140320+0000 mon.a (mon.0) 3383 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[18597]: cluster 2024-09-10T18:51:36.313201+0000 mon.a (mon.0) 3384 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[18597]: audit 2024-09-10T18:51:36.323962+0000 mon.a (mon.0) 3385 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[18597]: cluster 2024-09-10T18:51:36.324113+0000 mon.a (mon.0) 3386 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[18597]: audit 2024-09-10T18:51:36.325078+0000 mon.a (mon.0) 3387 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[31607]: cluster 2024-09-10T18:51:36.140320+0000 mon.a (mon.0) 3383 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[31607]: cluster 2024-09-10T18:51:36.313201+0000 mon.a (mon.0) 3384 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[31607]: audit 2024-09-10T18:51:36.323962+0000 mon.a (mon.0) 3385 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:51:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[31607]: cluster 2024-09-10T18:51:36.324113+0000 mon.a (mon.0) 3386 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-09-10T18:51:36.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:36 smithi100 bash[31607]: audit 2024-09-10T18:51:36.325078+0000 mon.a (mon.0) 3387 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:51:37.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:37 smithi151 bash[22565]: cluster 2024-09-10T18:51:36.206287+0000 mgr.y (mgr.24512) 664 : cluster [DBG] pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:51:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:37 smithi151 bash[22565]: audit 2024-09-10T18:51:37.327194+0000 mon.a (mon.0) 3388 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:51:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:37 smithi151 bash[22565]: cluster 2024-09-10T18:51:37.327357+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-09-10T18:51:37.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:37 smithi151 bash[22565]: audit 2024-09-10T18:51:37.328554+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:51:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[18597]: cluster 2024-09-10T18:51:36.206287+0000 mgr.y (mgr.24512) 664 : cluster [DBG] pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:51:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[18597]: audit 2024-09-10T18:51:37.327194+0000 mon.a (mon.0) 3388 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:51:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[18597]: cluster 2024-09-10T18:51:37.327357+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-09-10T18:51:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[18597]: audit 2024-09-10T18:51:37.328554+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:51:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[31607]: cluster 2024-09-10T18:51:36.206287+0000 mgr.y (mgr.24512) 664 : cluster [DBG] pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:51:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[31607]: audit 2024-09-10T18:51:37.327194+0000 mon.a (mon.0) 3388 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:51:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[31607]: cluster 2024-09-10T18:51:37.327357+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-09-10T18:51:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:37 smithi100 bash[31607]: audit 2024-09-10T18:51:37.328554+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:51:39.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:39 smithi151 bash[22565]: cluster 2024-09-10T18:51:38.207069+0000 mgr.y (mgr.24512) 665 : cluster [DBG] pgmap v1136: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:51:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:39 smithi151 bash[22565]: audit 2024-09-10T18:51:38.330377+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:51:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:39 smithi151 bash[22565]: cluster 2024-09-10T18:51:38.330676+0000 mon.a (mon.0) 3392 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-09-10T18:51:39.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:39 smithi151 bash[22565]: audit 2024-09-10T18:51:38.332645+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:51:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[18597]: cluster 2024-09-10T18:51:38.207069+0000 mgr.y (mgr.24512) 665 : cluster [DBG] pgmap v1136: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:51:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[18597]: audit 2024-09-10T18:51:38.330377+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:51:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[18597]: cluster 2024-09-10T18:51:38.330676+0000 mon.a (mon.0) 3392 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-09-10T18:51:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[18597]: audit 2024-09-10T18:51:38.332645+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:51:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[31607]: cluster 2024-09-10T18:51:38.207069+0000 mgr.y (mgr.24512) 665 : cluster [DBG] pgmap v1136: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:51:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[31607]: audit 2024-09-10T18:51:38.330377+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:51:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[31607]: cluster 2024-09-10T18:51:38.330676+0000 mon.a (mon.0) 3392 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-09-10T18:51:39.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:39 smithi100 bash[31607]: audit 2024-09-10T18:51:38.332645+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:51:40.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:40 smithi151 bash[22565]: audit 2024-09-10T18:51:39.331444+0000 mon.a (mon.0) 3394 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:51:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:40 smithi151 bash[22565]: cluster 2024-09-10T18:51:39.331609+0000 mon.a (mon.0) 3395 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-09-10T18:51:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:40 smithi151 bash[22565]: audit 2024-09-10T18:51:39.342034+0000 mon.c (mon.2) 520 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:40.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:40 smithi151 bash[22565]: audit 2024-09-10T18:51:39.400229+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[18597]: audit 2024-09-10T18:51:39.331444+0000 mon.a (mon.0) 3394 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:51:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[18597]: cluster 2024-09-10T18:51:39.331609+0000 mon.a (mon.0) 3395 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-09-10T18:51:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[18597]: audit 2024-09-10T18:51:39.342034+0000 mon.c (mon.2) 520 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:40.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[18597]: audit 2024-09-10T18:51:39.400229+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[31607]: audit 2024-09-10T18:51:39.331444+0000 mon.a (mon.0) 3394 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:51:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[31607]: cluster 2024-09-10T18:51:39.331609+0000 mon.a (mon.0) 3395 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-09-10T18:51:40.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[31607]: audit 2024-09-10T18:51:39.342034+0000 mon.c (mon.2) 520 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:40.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:40 smithi100 bash[31607]: audit 2024-09-10T18:51:39.400229+0000 mon.a (mon.0) 3396 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:41.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:41 smithi151 bash[22565]: cluster 2024-09-10T18:51:40.207878+0000 mgr.y (mgr.24512) 666 : cluster [DBG] pgmap v1139: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:41 smithi151 bash[22565]: audit 2024-09-10T18:51:40.354226+0000 mon.a (mon.0) 3397 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:51:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:41 smithi151 bash[22565]: cluster 2024-09-10T18:51:40.354396+0000 mon.a (mon.0) 3398 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-09-10T18:51:41.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:41 smithi151 bash[22565]: audit 2024-09-10T18:51:40.359526+0000 mon.a (mon.0) 3399 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:41] "GET /metrics HTTP/1.1" 200 228717 "" "Prometheus/2.43.0" 2024-09-10T18:51:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[18597]: cluster 2024-09-10T18:51:40.207878+0000 mgr.y (mgr.24512) 666 : cluster [DBG] pgmap v1139: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[18597]: audit 2024-09-10T18:51:40.354226+0000 mon.a (mon.0) 3397 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:51:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[18597]: cluster 2024-09-10T18:51:40.354396+0000 mon.a (mon.0) 3398 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-09-10T18:51:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[18597]: audit 2024-09-10T18:51:40.359526+0000 mon.a (mon.0) 3399 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[31607]: cluster 2024-09-10T18:51:40.207878+0000 mgr.y (mgr.24512) 666 : cluster [DBG] pgmap v1139: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[31607]: audit 2024-09-10T18:51:40.354226+0000 mon.a (mon.0) 3397 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:51:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[31607]: cluster 2024-09-10T18:51:40.354396+0000 mon.a (mon.0) 3398 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-09-10T18:51:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:41 smithi100 bash[31607]: audit 2024-09-10T18:51:40.359526+0000 mon.a (mon.0) 3399 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:42.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:42 smithi151 bash[22565]: audit 2024-09-10T18:51:41.359542+0000 mon.a (mon.0) 3400 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]': finished 2024-09-10T18:51:42.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:42 smithi151 bash[22565]: cluster 2024-09-10T18:51:41.359656+0000 mon.a (mon.0) 3401 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-09-10T18:51:42.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:42 smithi151 bash[22565]: audit 2024-09-10T18:51:41.399484+0000 mon.a (mon.0) 3402 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:42.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:42 smithi151 bash[22565]: audit 2024-09-10T18:51:41.400203+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:42.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[18597]: audit 2024-09-10T18:51:41.359542+0000 mon.a (mon.0) 3400 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]': finished 2024-09-10T18:51:42.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[18597]: cluster 2024-09-10T18:51:41.359656+0000 mon.a (mon.0) 3401 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-09-10T18:51:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[18597]: audit 2024-09-10T18:51:41.399484+0000 mon.a (mon.0) 3402 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:42.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[18597]: audit 2024-09-10T18:51:41.400203+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[31607]: audit 2024-09-10T18:51:41.359542+0000 mon.a (mon.0) 3400 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]': finished 2024-09-10T18:51:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[31607]: cluster 2024-09-10T18:51:41.359656+0000 mon.a (mon.0) 3401 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-09-10T18:51:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[31607]: audit 2024-09-10T18:51:41.399484+0000 mon.a (mon.0) 3402 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:42.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:42 smithi100 bash[31607]: audit 2024-09-10T18:51:41.400203+0000 mon.a (mon.0) 3403 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-142"}]: dispatch 2024-09-10T18:51:43.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:51:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:51:43.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:43 smithi151 bash[22565]: cluster 2024-09-10T18:51:42.208699+0000 mgr.y (mgr.24512) 667 : cluster [DBG] pgmap v1142: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:51:43.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:43 smithi151 bash[22565]: cluster 2024-09-10T18:51:42.363561+0000 mon.a (mon.0) 3404 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:43.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:43 smithi151 bash[22565]: cluster 2024-09-10T18:51:42.378492+0000 mon.a (mon.0) 3405 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-09-10T18:51:43.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:43 smithi151 bash[22565]: audit 2024-09-10T18:51:42.857726+0000 mgr.y (mgr.24512) 668 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[18597]: cluster 2024-09-10T18:51:42.208699+0000 mgr.y (mgr.24512) 667 : cluster [DBG] pgmap v1142: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:51:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[18597]: cluster 2024-09-10T18:51:42.363561+0000 mon.a (mon.0) 3404 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[18597]: cluster 2024-09-10T18:51:42.378492+0000 mon.a (mon.0) 3405 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-09-10T18:51:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[18597]: audit 2024-09-10T18:51:42.857726+0000 mgr.y (mgr.24512) 668 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[31607]: cluster 2024-09-10T18:51:42.208699+0000 mgr.y (mgr.24512) 667 : cluster [DBG] pgmap v1142: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:51:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[31607]: cluster 2024-09-10T18:51:42.363561+0000 mon.a (mon.0) 3404 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:43.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[31607]: cluster 2024-09-10T18:51:42.378492+0000 mon.a (mon.0) 3405 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-09-10T18:51:43.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:43 smithi100 bash[31607]: audit 2024-09-10T18:51:42.857726+0000 mgr.y (mgr.24512) 668 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:44.671 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:51:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:44.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:44 smithi151 bash[22565]: cluster 2024-09-10T18:51:43.384220+0000 mon.a (mon.0) 3406 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-09-10T18:51:44.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:44 smithi151 bash[22565]: audit 2024-09-10T18:51:43.386111+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:44.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:44 smithi100 bash[18597]: cluster 2024-09-10T18:51:43.384220+0000 mon.a (mon.0) 3406 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-09-10T18:51:44.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:44 smithi100 bash[18597]: audit 2024-09-10T18:51:43.386111+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:44 smithi100 bash[31607]: cluster 2024-09-10T18:51:43.384220+0000 mon.a (mon.0) 3406 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-09-10T18:51:44.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:44 smithi100 bash[31607]: audit 2024-09-10T18:51:43.386111+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:45.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:45 smithi151 bash[22565]: cluster 2024-09-10T18:51:44.209506+0000 mgr.y (mgr.24512) 669 : cluster [DBG] pgmap v1145: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:51:45.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:45 smithi151 bash[22565]: audit 2024-09-10T18:51:44.389757+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:45.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:45 smithi151 bash[22565]: cluster 2024-09-10T18:51:44.390311+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-09-10T18:51:45.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:45 smithi151 bash[22565]: audit 2024-09-10T18:51:44.415049+0000 mon.a (mon.0) 3410 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[18597]: cluster 2024-09-10T18:51:44.209506+0000 mgr.y (mgr.24512) 669 : cluster [DBG] pgmap v1145: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[18597]: audit 2024-09-10T18:51:44.389757+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[18597]: cluster 2024-09-10T18:51:44.390311+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[18597]: audit 2024-09-10T18:51:44.415049+0000 mon.a (mon.0) 3410 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[31607]: cluster 2024-09-10T18:51:44.209506+0000 mgr.y (mgr.24512) 669 : cluster [DBG] pgmap v1145: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[31607]: audit 2024-09-10T18:51:44.389757+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[31607]: cluster 2024-09-10T18:51:44.390311+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-09-10T18:51:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:45 smithi100 bash[31607]: audit 2024-09-10T18:51:44.415049+0000 mon.a (mon.0) 3410 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:46 smithi100 bash[18597]: audit 2024-09-10T18:51:45.415256+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:46 smithi100 bash[18597]: cluster 2024-09-10T18:51:45.415306+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-09-10T18:51:46.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:46 smithi100 bash[18597]: audit 2024-09-10T18:51:45.416059+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:51:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:46 smithi100 bash[31607]: audit 2024-09-10T18:51:45.415256+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:46 smithi100 bash[31607]: cluster 2024-09-10T18:51:45.415306+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-09-10T18:51:46.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:46 smithi100 bash[31607]: audit 2024-09-10T18:51:45.416059+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:51:46.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:46 smithi151 bash[22565]: audit 2024-09-10T18:51:45.415256+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:46.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:46 smithi151 bash[22565]: cluster 2024-09-10T18:51:45.415306+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-09-10T18:51:46.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:46 smithi151 bash[22565]: audit 2024-09-10T18:51:45.416059+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[18597]: cluster 2024-09-10T18:51:46.210239+0000 mgr.y (mgr.24512) 670 : cluster [DBG] pgmap v1148: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[18597]: audit 2024-09-10T18:51:46.421083+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-144"}]': finished 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[18597]: cluster 2024-09-10T18:51:46.421285+0000 mon.a (mon.0) 3415 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[18597]: audit 2024-09-10T18:51:46.423143+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-144", "mode": "readproxy"}]: dispatch 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[18597]: cluster 2024-09-10T18:51:47.414127+0000 mon.a (mon.0) 3417 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[31607]: cluster 2024-09-10T18:51:46.210239+0000 mgr.y (mgr.24512) 670 : cluster [DBG] pgmap v1148: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[31607]: audit 2024-09-10T18:51:46.421083+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-144"}]': finished 2024-09-10T18:51:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[31607]: cluster 2024-09-10T18:51:46.421285+0000 mon.a (mon.0) 3415 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-09-10T18:51:47.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[31607]: audit 2024-09-10T18:51:46.423143+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-144", "mode": "readproxy"}]: dispatch 2024-09-10T18:51:47.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:47 smithi100 bash[31607]: cluster 2024-09-10T18:51:47.414127+0000 mon.a (mon.0) 3417 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:47 smithi151 bash[22565]: cluster 2024-09-10T18:51:46.210239+0000 mgr.y (mgr.24512) 670 : cluster [DBG] pgmap v1148: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:47 smithi151 bash[22565]: audit 2024-09-10T18:51:46.421083+0000 mon.a (mon.0) 3414 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-144"}]': finished 2024-09-10T18:51:47.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:47 smithi151 bash[22565]: cluster 2024-09-10T18:51:46.421285+0000 mon.a (mon.0) 3415 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-09-10T18:51:47.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:47 smithi151 bash[22565]: audit 2024-09-10T18:51:46.423143+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-144", "mode": "readproxy"}]: dispatch 2024-09-10T18:51:47.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:47 smithi151 bash[22565]: cluster 2024-09-10T18:51:47.414127+0000 mon.a (mon.0) 3417 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:48 smithi100 bash[18597]: audit 2024-09-10T18:51:47.425680+0000 mon.a (mon.0) 3418 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-144", "mode": "readproxy"}]': finished 2024-09-10T18:51:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:48 smithi100 bash[18597]: cluster 2024-09-10T18:51:47.425826+0000 mon.a (mon.0) 3419 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-09-10T18:51:48.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:48 smithi100 bash[18597]: audit 2024-09-10T18:51:48.338826+0000 mon.c (mon.2) 521 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:48 smithi100 bash[31607]: audit 2024-09-10T18:51:47.425680+0000 mon.a (mon.0) 3418 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-144", "mode": "readproxy"}]': finished 2024-09-10T18:51:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:48 smithi100 bash[31607]: cluster 2024-09-10T18:51:47.425826+0000 mon.a (mon.0) 3419 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-09-10T18:51:48.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:48 smithi100 bash[31607]: audit 2024-09-10T18:51:48.338826+0000 mon.c (mon.2) 521 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:48.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:48 smithi151 bash[22565]: audit 2024-09-10T18:51:47.425680+0000 mon.a (mon.0) 3418 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-144", "mode": "readproxy"}]': finished 2024-09-10T18:51:48.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:48 smithi151 bash[22565]: cluster 2024-09-10T18:51:47.425826+0000 mon.a (mon.0) 3419 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-09-10T18:51:48.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:48 smithi151 bash[22565]: audit 2024-09-10T18:51:48.338826+0000 mon.c (mon.2) 521 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[18597]: cluster 2024-09-10T18:51:48.211056+0000 mgr.y (mgr.24512) 671 : cluster [DBG] pgmap v1151: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[18597]: audit 2024-09-10T18:51:48.799662+0000 mon.c (mon.2) 522 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[18597]: audit 2024-09-10T18:51:48.800863+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[18597]: audit 2024-09-10T18:51:48.810843+0000 mon.a (mon.0) 3420 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[31607]: cluster 2024-09-10T18:51:48.211056+0000 mgr.y (mgr.24512) 671 : cluster [DBG] pgmap v1151: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[31607]: audit 2024-09-10T18:51:48.799662+0000 mon.c (mon.2) 522 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[31607]: audit 2024-09-10T18:51:48.800863+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:49 smithi100 bash[31607]: audit 2024-09-10T18:51:48.810843+0000 mon.a (mon.0) 3420 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:51:49.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:49 smithi151 bash[22565]: cluster 2024-09-10T18:51:48.211056+0000 mgr.y (mgr.24512) 671 : cluster [DBG] pgmap v1151: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:49.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:49 smithi151 bash[22565]: audit 2024-09-10T18:51:48.799662+0000 mon.c (mon.2) 522 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:49.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:49 smithi151 bash[22565]: audit 2024-09-10T18:51:48.800863+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:49.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:49 smithi151 bash[22565]: audit 2024-09-10T18:51:48.810843+0000 mon.a (mon.0) 3420 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:51:51.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:51:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:51] "GET /metrics HTTP/1.1" 200 228717 "" "Prometheus/2.43.0" 2024-09-10T18:51:51.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:51 smithi100 bash[18597]: cluster 2024-09-10T18:51:50.211983+0000 mgr.y (mgr.24512) 672 : cluster [DBG] pgmap v1152: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 878 B/s rd, 175 B/s wr, 1 op/s 2024-09-10T18:51:51.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:51 smithi100 bash[18597]: cluster 2024-09-10T18:51:51.148799+0000 mon.a (mon.0) 3421 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:51 smithi100 bash[31607]: cluster 2024-09-10T18:51:50.211983+0000 mgr.y (mgr.24512) 672 : cluster [DBG] pgmap v1152: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 878 B/s rd, 175 B/s wr, 1 op/s 2024-09-10T18:51:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:51 smithi100 bash[31607]: cluster 2024-09-10T18:51:51.148799+0000 mon.a (mon.0) 3421 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:51.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:51 smithi151 bash[22565]: cluster 2024-09-10T18:51:50.211983+0000 mgr.y (mgr.24512) 672 : cluster [DBG] pgmap v1152: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 878 B/s rd, 175 B/s wr, 1 op/s 2024-09-10T18:51:51.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:51 smithi151 bash[22565]: cluster 2024-09-10T18:51:51.148799+0000 mon.a (mon.0) 3421 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:53.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:51:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:51:53.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:53 smithi100 bash[18597]: cluster 2024-09-10T18:51:52.214125+0000 mgr.y (mgr.24512) 673 : cluster [DBG] pgmap v1153: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 903 B/s rd, 0 op/s 2024-09-10T18:51:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:53 smithi100 bash[18597]: audit 2024-09-10T18:51:52.869229+0000 mgr.y (mgr.24512) 674 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:53 smithi100 bash[31607]: cluster 2024-09-10T18:51:52.214125+0000 mgr.y (mgr.24512) 673 : cluster [DBG] pgmap v1153: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 903 B/s rd, 0 op/s 2024-09-10T18:51:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:53 smithi100 bash[31607]: audit 2024-09-10T18:51:52.869229+0000 mgr.y (mgr.24512) 674 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:53.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:53 smithi151 bash[22565]: cluster 2024-09-10T18:51:52.214125+0000 mgr.y (mgr.24512) 673 : cluster [DBG] pgmap v1153: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 903 B/s rd, 0 op/s 2024-09-10T18:51:53.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:53 smithi151 bash[22565]: audit 2024-09-10T18:51:52.869229+0000 mgr.y (mgr.24512) 674 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:54.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:51:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:51:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:55 smithi151 bash[22565]: cluster 2024-09-10T18:51:54.214955+0000 mgr.y (mgr.24512) 675 : cluster [DBG] pgmap v1154: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:51:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:55 smithi151 bash[22565]: audit 2024-09-10T18:51:54.349954+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:51:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:55 smithi151 bash[22565]: audit 2024-09-10T18:51:54.351911+0000 mon.c (mon.2) 524 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:55.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:55 smithi100 bash[18597]: cluster 2024-09-10T18:51:54.214955+0000 mgr.y (mgr.24512) 675 : cluster [DBG] pgmap v1154: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:51:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:55 smithi100 bash[18597]: audit 2024-09-10T18:51:54.349954+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:51:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:55 smithi100 bash[18597]: audit 2024-09-10T18:51:54.351911+0000 mon.c (mon.2) 524 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:55 smithi100 bash[31607]: cluster 2024-09-10T18:51:54.214955+0000 mgr.y (mgr.24512) 675 : cluster [DBG] pgmap v1154: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:51:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:55 smithi100 bash[31607]: audit 2024-09-10T18:51:54.349954+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:51:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:55 smithi100 bash[31607]: audit 2024-09-10T18:51:54.351911+0000 mon.c (mon.2) 524 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:57.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:57 smithi151 bash[22565]: cluster 2024-09-10T18:51:56.217065+0000 mgr.y (mgr.24512) 676 : cluster [DBG] pgmap v1155: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:51:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:57 smithi100 bash[18597]: cluster 2024-09-10T18:51:56.217065+0000 mgr.y (mgr.24512) 676 : cluster [DBG] pgmap v1155: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:51:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:57 smithi100 bash[31607]: cluster 2024-09-10T18:51:56.217065+0000 mgr.y (mgr.24512) 676 : cluster [DBG] pgmap v1155: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:51:58.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:58 smithi151 bash[22565]: audit 2024-09-10T18:51:57.697064+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:58.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:58 smithi100 bash[18597]: audit 2024-09-10T18:51:57.697064+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:58.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:58 smithi100 bash[31607]: audit 2024-09-10T18:51:57.697064+0000 mon.a (mon.0) 3423 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:51:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:59 smithi151 bash[22565]: cluster 2024-09-10T18:51:58.218008+0000 mgr.y (mgr.24512) 677 : cluster [DBG] pgmap v1156: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:59.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:59 smithi151 bash[22565]: audit 2024-09-10T18:51:58.368062+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:51:59.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:59 smithi151 bash[22565]: cluster 2024-09-10T18:51:58.368145+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-09-10T18:51:59.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:51:59 smithi151 bash[22565]: audit 2024-09-10T18:51:58.369197+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:51:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[18597]: cluster 2024-09-10T18:51:58.218008+0000 mgr.y (mgr.24512) 677 : cluster [DBG] pgmap v1156: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[18597]: audit 2024-09-10T18:51:58.368062+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:51:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[18597]: cluster 2024-09-10T18:51:58.368145+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-09-10T18:51:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[18597]: audit 2024-09-10T18:51:58.369197+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:51:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[31607]: cluster 2024-09-10T18:51:58.218008+0000 mgr.y (mgr.24512) 677 : cluster [DBG] pgmap v1156: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:51:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[31607]: audit 2024-09-10T18:51:58.368062+0000 mon.a (mon.0) 3424 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:51:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[31607]: cluster 2024-09-10T18:51:58.368145+0000 mon.a (mon.0) 3425 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-09-10T18:51:59.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:51:59 smithi100 bash[31607]: audit 2024-09-10T18:51:58.369197+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:52:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:00 smithi151 bash[22565]: cluster 2024-09-10T18:51:59.361585+0000 mon.a (mon.0) 3427 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:00 smithi151 bash[22565]: audit 2024-09-10T18:51:59.371227+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]': finished 2024-09-10T18:52:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:00 smithi151 bash[22565]: cluster 2024-09-10T18:51:59.371932+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-09-10T18:52:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:00 smithi151 bash[22565]: audit 2024-09-10T18:51:59.413207+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:00.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:00 smithi151 bash[22565]: audit 2024-09-10T18:51:59.413915+0000 mon.a (mon.0) 3431 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:52:00.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[18597]: cluster 2024-09-10T18:51:59.361585+0000 mon.a (mon.0) 3427 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[18597]: audit 2024-09-10T18:51:59.371227+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]': finished 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[18597]: cluster 2024-09-10T18:51:59.371932+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[18597]: audit 2024-09-10T18:51:59.413207+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[18597]: audit 2024-09-10T18:51:59.413915+0000 mon.a (mon.0) 3431 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[31607]: cluster 2024-09-10T18:51:59.361585+0000 mon.a (mon.0) 3427 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[31607]: audit 2024-09-10T18:51:59.371227+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]': finished 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[31607]: cluster 2024-09-10T18:51:59.371932+0000 mon.a (mon.0) 3429 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-09-10T18:52:00.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[31607]: audit 2024-09-10T18:51:59.413207+0000 mon.a (mon.0) 3430 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:00.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:00 smithi100 bash[31607]: audit 2024-09-10T18:51:59.413915+0000 mon.a (mon.0) 3431 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-144"}]: dispatch 2024-09-10T18:52:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:01 smithi151 bash[22565]: cluster 2024-09-10T18:52:00.218847+0000 mgr.y (mgr.24512) 678 : cluster [DBG] pgmap v1159: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:52:01.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:01 smithi151 bash[22565]: cluster 2024-09-10T18:52:00.375889+0000 mon.a (mon.0) 3432 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:01.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:01 smithi151 bash[22565]: cluster 2024-09-10T18:52:00.391150+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-09-10T18:52:01.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:52:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:01] "GET /metrics HTTP/1.1" 200 228721 "" "Prometheus/2.43.0" 2024-09-10T18:52:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:01 smithi100 bash[18597]: cluster 2024-09-10T18:52:00.218847+0000 mgr.y (mgr.24512) 678 : cluster [DBG] pgmap v1159: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:52:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:01 smithi100 bash[18597]: cluster 2024-09-10T18:52:00.375889+0000 mon.a (mon.0) 3432 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:01.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:01 smithi100 bash[18597]: cluster 2024-09-10T18:52:00.391150+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-09-10T18:52:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:01 smithi100 bash[31607]: cluster 2024-09-10T18:52:00.218847+0000 mgr.y (mgr.24512) 678 : cluster [DBG] pgmap v1159: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:52:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:01 smithi100 bash[31607]: cluster 2024-09-10T18:52:00.375889+0000 mon.a (mon.0) 3432 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:01 smithi100 bash[31607]: cluster 2024-09-10T18:52:00.391150+0000 mon.a (mon.0) 3433 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-09-10T18:52:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:02 smithi151 bash[22565]: cluster 2024-09-10T18:52:01.397442+0000 mon.a (mon.0) 3434 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-09-10T18:52:02.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:02 smithi151 bash[22565]: audit 2024-09-10T18:52:01.403721+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:02.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:02 smithi100 bash[18597]: cluster 2024-09-10T18:52:01.397442+0000 mon.a (mon.0) 3434 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-09-10T18:52:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:02 smithi100 bash[18597]: audit 2024-09-10T18:52:01.403721+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:02 smithi100 bash[31607]: cluster 2024-09-10T18:52:01.397442+0000 mon.a (mon.0) 3434 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-09-10T18:52:02.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:02 smithi100 bash[31607]: audit 2024-09-10T18:52:01.403721+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:03.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:52:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:52:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:03 smithi151 bash[22565]: cluster 2024-09-10T18:52:02.219662+0000 mgr.y (mgr.24512) 679 : cluster [DBG] pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:03 smithi151 bash[22565]: audit 2024-09-10T18:52:02.408724+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:03 smithi151 bash[22565]: cluster 2024-09-10T18:52:02.408969+0000 mon.a (mon.0) 3437 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-09-10T18:52:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:03 smithi151 bash[22565]: audit 2024-09-10T18:52:02.440188+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:03 smithi151 bash[22565]: audit 2024-09-10T18:52:02.880123+0000 mgr.y (mgr.24512) 680 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[18597]: cluster 2024-09-10T18:52:02.219662+0000 mgr.y (mgr.24512) 679 : cluster [DBG] pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[18597]: audit 2024-09-10T18:52:02.408724+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[18597]: cluster 2024-09-10T18:52:02.408969+0000 mon.a (mon.0) 3437 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[18597]: audit 2024-09-10T18:52:02.440188+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[18597]: audit 2024-09-10T18:52:02.880123+0000 mgr.y (mgr.24512) 680 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[31607]: cluster 2024-09-10T18:52:02.219662+0000 mgr.y (mgr.24512) 679 : cluster [DBG] pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[31607]: audit 2024-09-10T18:52:02.408724+0000 mon.a (mon.0) 3436 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[31607]: cluster 2024-09-10T18:52:02.408969+0000 mon.a (mon.0) 3437 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-09-10T18:52:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[31607]: audit 2024-09-10T18:52:02.440188+0000 mon.a (mon.0) 3438 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:03 smithi100 bash[31607]: audit 2024-09-10T18:52:02.880123+0000 mgr.y (mgr.24512) 680 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:04.671 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:52:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:04.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:04 smithi151 bash[22565]: audit 2024-09-10T18:52:03.412818+0000 mon.a (mon.0) 3439 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:04.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:04 smithi151 bash[22565]: cluster 2024-09-10T18:52:03.412966+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-09-10T18:52:04.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:04 smithi151 bash[22565]: audit 2024-09-10T18:52:03.415011+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:04.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:04 smithi151 bash[22565]: audit 2024-09-10T18:52:04.416418+0000 mon.a (mon.0) 3442 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-146"}]': finished 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[18597]: audit 2024-09-10T18:52:03.412818+0000 mon.a (mon.0) 3439 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[18597]: cluster 2024-09-10T18:52:03.412966+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[18597]: audit 2024-09-10T18:52:03.415011+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[18597]: audit 2024-09-10T18:52:04.416418+0000 mon.a (mon.0) 3442 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-146"}]': finished 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[31607]: audit 2024-09-10T18:52:03.412818+0000 mon.a (mon.0) 3439 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[31607]: cluster 2024-09-10T18:52:03.412966+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[31607]: audit 2024-09-10T18:52:03.415011+0000 mon.a (mon.0) 3441 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:04 smithi100 bash[31607]: audit 2024-09-10T18:52:04.416418+0000 mon.a (mon.0) 3442 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-72764-111", "overlaypool": "test-rados-api-smithi100-72764-146"}]': finished 2024-09-10T18:52:05.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[18597]: cluster 2024-09-10T18:52:04.220485+0000 mgr.y (mgr.24512) 681 : cluster [DBG] pgmap v1165: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[18597]: cluster 2024-09-10T18:52:04.416575+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-09-10T18:52:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[18597]: audit 2024-09-10T18:52:04.419424+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-146", "mode": "writeback"}]: dispatch 2024-09-10T18:52:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[18597]: cluster 2024-09-10T18:52:05.409895+0000 mon.a (mon.0) 3445 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[31607]: cluster 2024-09-10T18:52:04.220485+0000 mgr.y (mgr.24512) 681 : cluster [DBG] pgmap v1165: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[31607]: cluster 2024-09-10T18:52:04.416575+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-09-10T18:52:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[31607]: audit 2024-09-10T18:52:04.419424+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-146", "mode": "writeback"}]: dispatch 2024-09-10T18:52:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:05 smithi100 bash[31607]: cluster 2024-09-10T18:52:05.409895+0000 mon.a (mon.0) 3445 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:05.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:05 smithi151 bash[22565]: cluster 2024-09-10T18:52:04.220485+0000 mgr.y (mgr.24512) 681 : cluster [DBG] pgmap v1165: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:05.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:05 smithi151 bash[22565]: cluster 2024-09-10T18:52:04.416575+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-09-10T18:52:05.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:05 smithi151 bash[22565]: audit 2024-09-10T18:52:04.419424+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-146", "mode": "writeback"}]: dispatch 2024-09-10T18:52:05.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:05 smithi151 bash[22565]: cluster 2024-09-10T18:52:05.409895+0000 mon.a (mon.0) 3445 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[18597]: audit 2024-09-10T18:52:05.418580+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-146", "mode": "writeback"}]': finished 2024-09-10T18:52:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[18597]: cluster 2024-09-10T18:52:05.418747+0000 mon.a (mon.0) 3447 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-09-10T18:52:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[18597]: audit 2024-09-10T18:52:05.493088+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:52:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[18597]: cluster 2024-09-10T18:52:06.153411+0000 mon.a (mon.0) 3449 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[31607]: audit 2024-09-10T18:52:05.418580+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-146", "mode": "writeback"}]': finished 2024-09-10T18:52:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[31607]: cluster 2024-09-10T18:52:05.418747+0000 mon.a (mon.0) 3447 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-09-10T18:52:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[31607]: audit 2024-09-10T18:52:05.493088+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:52:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:06 smithi100 bash[31607]: cluster 2024-09-10T18:52:06.153411+0000 mon.a (mon.0) 3449 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:06.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:06 smithi151 bash[22565]: audit 2024-09-10T18:52:05.418580+0000 mon.a (mon.0) 3446 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-72764-146", "mode": "writeback"}]': finished 2024-09-10T18:52:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:06 smithi151 bash[22565]: cluster 2024-09-10T18:52:05.418747+0000 mon.a (mon.0) 3447 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-09-10T18:52:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:06 smithi151 bash[22565]: audit 2024-09-10T18:52:05.493088+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:52:06.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:06 smithi151 bash[22565]: cluster 2024-09-10T18:52:06.153411+0000 mon.a (mon.0) 3449 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:07.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[18597]: cluster 2024-09-10T18:52:06.221307+0000 mgr.y (mgr.24512) 682 : cluster [DBG] pgmap v1168: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:52:07.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[18597]: audit 2024-09-10T18:52:06.446649+0000 mon.a (mon.0) 3450 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:52:07.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[18597]: cluster 2024-09-10T18:52:06.446736+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-09-10T18:52:07.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[18597]: audit 2024-09-10T18:52:06.448019+0000 mon.a (mon.0) 3452 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:52:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[31607]: cluster 2024-09-10T18:52:06.221307+0000 mgr.y (mgr.24512) 682 : cluster [DBG] pgmap v1168: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:52:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[31607]: audit 2024-09-10T18:52:06.446649+0000 mon.a (mon.0) 3450 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:52:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[31607]: cluster 2024-09-10T18:52:06.446736+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-09-10T18:52:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:07 smithi100 bash[31607]: audit 2024-09-10T18:52:06.448019+0000 mon.a (mon.0) 3452 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:52:07.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:07 smithi151 bash[22565]: cluster 2024-09-10T18:52:06.221307+0000 mgr.y (mgr.24512) 682 : cluster [DBG] pgmap v1168: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:52:07.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:07 smithi151 bash[22565]: audit 2024-09-10T18:52:06.446649+0000 mon.a (mon.0) 3450 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:52:07.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:07 smithi151 bash[22565]: cluster 2024-09-10T18:52:06.446736+0000 mon.a (mon.0) 3451 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-09-10T18:52:07.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:07 smithi151 bash[22565]: audit 2024-09-10T18:52:06.448019+0000 mon.a (mon.0) 3452 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:52:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:08 smithi100 bash[18597]: audit 2024-09-10T18:52:07.456071+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:52:08.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:08 smithi100 bash[18597]: cluster 2024-09-10T18:52:07.456260+0000 mon.a (mon.0) 3454 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-09-10T18:52:08.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:08 smithi100 bash[18597]: audit 2024-09-10T18:52:07.457269+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:52:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:08 smithi100 bash[31607]: audit 2024-09-10T18:52:07.456071+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:52:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:08 smithi100 bash[31607]: cluster 2024-09-10T18:52:07.456260+0000 mon.a (mon.0) 3454 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-09-10T18:52:08.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:08 smithi100 bash[31607]: audit 2024-09-10T18:52:07.457269+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:52:08.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:08 smithi151 bash[22565]: audit 2024-09-10T18:52:07.456071+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:52:08.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:08 smithi151 bash[22565]: cluster 2024-09-10T18:52:07.456260+0000 mon.a (mon.0) 3454 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-09-10T18:52:08.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:08 smithi151 bash[22565]: audit 2024-09-10T18:52:07.457269+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[18597]: cluster 2024-09-10T18:52:08.222246+0000 mgr.y (mgr.24512) 683 : cluster [DBG] pgmap v1171: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[18597]: cluster 2024-09-10T18:52:08.451865+0000 mon.a (mon.0) 3456 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[18597]: audit 2024-09-10T18:52:08.468639+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[18597]: cluster 2024-09-10T18:52:08.468743+0000 mon.a (mon.0) 3458 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[18597]: audit 2024-09-10T18:52:08.469370+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[18597]: audit 2024-09-10T18:52:09.350669+0000 mon.a (mon.0) 3460 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[18597]: audit 2024-09-10T18:52:09.353262+0000 mon.c (mon.2) 525 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[31607]: cluster 2024-09-10T18:52:08.222246+0000 mgr.y (mgr.24512) 683 : cluster [DBG] pgmap v1171: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:52:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[31607]: cluster 2024-09-10T18:52:08.451865+0000 mon.a (mon.0) 3456 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[31607]: audit 2024-09-10T18:52:08.468639+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:52:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[31607]: cluster 2024-09-10T18:52:08.468743+0000 mon.a (mon.0) 3458 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-09-10T18:52:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[31607]: audit 2024-09-10T18:52:08.469370+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:52:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[31607]: audit 2024-09-10T18:52:09.350669+0000 mon.a (mon.0) 3460 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:09.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:09 smithi100 bash[31607]: audit 2024-09-10T18:52:09.353262+0000 mon.c (mon.2) 525 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:09.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:09 smithi151 bash[22565]: cluster 2024-09-10T18:52:08.222246+0000 mgr.y (mgr.24512) 683 : cluster [DBG] pgmap v1171: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:52:09.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:09 smithi151 bash[22565]: cluster 2024-09-10T18:52:08.451865+0000 mon.a (mon.0) 3456 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:09.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:09 smithi151 bash[22565]: audit 2024-09-10T18:52:08.468639+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:52:09.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:09 smithi151 bash[22565]: cluster 2024-09-10T18:52:08.468743+0000 mon.a (mon.0) 3458 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-09-10T18:52:09.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:09 smithi151 bash[22565]: audit 2024-09-10T18:52:08.469370+0000 mon.a (mon.0) 3459 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:52:09.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:09 smithi151 bash[22565]: audit 2024-09-10T18:52:09.350669+0000 mon.a (mon.0) 3460 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:09.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:09 smithi151 bash[22565]: audit 2024-09-10T18:52:09.353262+0000 mon.c (mon.2) 525 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[18597]: audit 2024-09-10T18:52:09.468971+0000 mon.a (mon.0) 3461 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:52:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[18597]: cluster 2024-09-10T18:52:09.469077+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-09-10T18:52:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[18597]: audit 2024-09-10T18:52:09.470499+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:52:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[18597]: audit 2024-09-10T18:52:10.472488+0000 mon.a (mon.0) 3464 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:52:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[18597]: cluster 2024-09-10T18:52:10.472635+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-09-10T18:52:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[31607]: audit 2024-09-10T18:52:09.468971+0000 mon.a (mon.0) 3461 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:52:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[31607]: cluster 2024-09-10T18:52:09.469077+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-09-10T18:52:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[31607]: audit 2024-09-10T18:52:09.470499+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:52:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[31607]: audit 2024-09-10T18:52:10.472488+0000 mon.a (mon.0) 3464 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:52:10.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:10 smithi100 bash[31607]: cluster 2024-09-10T18:52:10.472635+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-09-10T18:52:10.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:10 smithi151 bash[22565]: audit 2024-09-10T18:52:09.468971+0000 mon.a (mon.0) 3461 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:52:10.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:10 smithi151 bash[22565]: cluster 2024-09-10T18:52:09.469077+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-09-10T18:52:10.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:10 smithi151 bash[22565]: audit 2024-09-10T18:52:09.470499+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:52:10.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:10 smithi151 bash[22565]: audit 2024-09-10T18:52:10.472488+0000 mon.a (mon.0) 3464 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:52:10.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:10 smithi151 bash[22565]: cluster 2024-09-10T18:52:10.472635+0000 mon.a (mon.0) 3465 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-09-10T18:52:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:11 smithi100 bash[18597]: cluster 2024-09-10T18:52:10.223096+0000 mgr.y (mgr.24512) 684 : cluster [DBG] pgmap v1174: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:52:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:11] "GET /metrics HTTP/1.1" 200 228723 "" "Prometheus/2.43.0" 2024-09-10T18:52:11.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:11 smithi100 bash[31607]: cluster 2024-09-10T18:52:10.223096+0000 mgr.y (mgr.24512) 684 : cluster [DBG] pgmap v1174: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:11.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:11 smithi151 bash[22565]: cluster 2024-09-10T18:52:10.223096+0000 mgr.y (mgr.24512) 684 : cluster [DBG] pgmap v1174: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:12.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:12 smithi100 bash[18597]: cluster 2024-09-10T18:52:12.487375+0000 mon.a (mon.0) 3466 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:52:12.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:12 smithi100 bash[31607]: cluster 2024-09-10T18:52:12.487375+0000 mon.a (mon.0) 3466 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:52:12.888 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:12 smithi151 bash[22565]: cluster 2024-09-10T18:52:12.487375+0000 mon.a (mon.0) 3466 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:52:13.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:52:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:52:13.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:13 smithi100 bash[18597]: cluster 2024-09-10T18:52:12.223939+0000 mgr.y (mgr.24512) 685 : cluster [DBG] pgmap v1176: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:52:13.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:13 smithi100 bash[18597]: audit 2024-09-10T18:52:12.888119+0000 mgr.y (mgr.24512) 686 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:13 smithi100 bash[31607]: cluster 2024-09-10T18:52:12.223939+0000 mgr.y (mgr.24512) 685 : cluster [DBG] pgmap v1176: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:52:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:13 smithi100 bash[31607]: audit 2024-09-10T18:52:12.888119+0000 mgr.y (mgr.24512) 686 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:13.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:13 smithi151 bash[22565]: cluster 2024-09-10T18:52:12.223939+0000 mgr.y (mgr.24512) 685 : cluster [DBG] pgmap v1176: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:52:13.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:13 smithi151 bash[22565]: audit 2024-09-10T18:52:12.888119+0000 mgr.y (mgr.24512) 686 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:14.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:52:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:15.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:15 smithi151 bash[22565]: cluster 2024-09-10T18:52:14.224889+0000 mgr.y (mgr.24512) 687 : cluster [DBG] pgmap v1177: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:16.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:15 smithi100 bash[31607]: cluster 2024-09-10T18:52:14.224889+0000 mgr.y (mgr.24512) 687 : cluster [DBG] pgmap v1177: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:16.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:15 smithi100 bash[18597]: cluster 2024-09-10T18:52:14.224889+0000 mgr.y (mgr.24512) 687 : cluster [DBG] pgmap v1177: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:17.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:17 smithi151 bash[22565]: cluster 2024-09-10T18:52:16.227057+0000 mgr.y (mgr.24512) 688 : cluster [DBG] pgmap v1178: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:18.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:17 smithi100 bash[31607]: cluster 2024-09-10T18:52:16.227057+0000 mgr.y (mgr.24512) 688 : cluster [DBG] pgmap v1178: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:18.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:17 smithi100 bash[18597]: cluster 2024-09-10T18:52:16.227057+0000 mgr.y (mgr.24512) 688 : cluster [DBG] pgmap v1178: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:19.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:19 smithi151 bash[22565]: cluster 2024-09-10T18:52:18.227889+0000 mgr.y (mgr.24512) 689 : cluster [DBG] pgmap v1179: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:20.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:19 smithi100 bash[31607]: cluster 2024-09-10T18:52:18.227889+0000 mgr.y (mgr.24512) 689 : cluster [DBG] pgmap v1179: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:20.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:19 smithi100 bash[18597]: cluster 2024-09-10T18:52:18.227889+0000 mgr.y (mgr.24512) 689 : cluster [DBG] pgmap v1179: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:20.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:20 smithi151 bash[22565]: audit 2024-09-10T18:52:20.497320+0000 mon.a (mon.0) 3467 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:21.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:20 smithi100 bash[31607]: audit 2024-09-10T18:52:20.497320+0000 mon.a (mon.0) 3467 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:21.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:20 smithi100 bash[18597]: audit 2024-09-10T18:52:20.497320+0000 mon.a (mon.0) 3467 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:21] "GET /metrics HTTP/1.1" 200 228723 "" "Prometheus/2.43.0" 2024-09-10T18:52:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[31607]: cluster 2024-09-10T18:52:20.228722+0000 mgr.y (mgr.24512) 690 : cluster [DBG] pgmap v1180: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[31607]: audit 2024-09-10T18:52:20.570368+0000 mon.a (mon.0) 3468 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[31607]: cluster 2024-09-10T18:52:20.570453+0000 mon.a (mon.0) 3469 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-09-10T18:52:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[31607]: audit 2024-09-10T18:52:20.572592+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[18597]: cluster 2024-09-10T18:52:20.228722+0000 mgr.y (mgr.24512) 690 : cluster [DBG] pgmap v1180: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[18597]: audit 2024-09-10T18:52:20.570368+0000 mon.a (mon.0) 3468 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:21.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[18597]: cluster 2024-09-10T18:52:20.570453+0000 mon.a (mon.0) 3469 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-09-10T18:52:21.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:21 smithi100 bash[18597]: audit 2024-09-10T18:52:20.572592+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:21.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:21 smithi151 bash[22565]: cluster 2024-09-10T18:52:20.228722+0000 mgr.y (mgr.24512) 690 : cluster [DBG] pgmap v1180: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:52:21.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:21 smithi151 bash[22565]: audit 2024-09-10T18:52:20.570368+0000 mon.a (mon.0) 3468 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:21.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:21 smithi151 bash[22565]: cluster 2024-09-10T18:52:20.570453+0000 mon.a (mon.0) 3469 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-09-10T18:52:21.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:21 smithi151 bash[22565]: audit 2024-09-10T18:52:20.572592+0000 mon.a (mon.0) 3470 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:22.900 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:22 smithi151 bash[22565]: audit 2024-09-10T18:52:21.575164+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]': finished 2024-09-10T18:52:22.900 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:22 smithi151 bash[22565]: cluster 2024-09-10T18:52:21.575335+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-09-10T18:52:22.900 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:22 smithi151 bash[22565]: audit 2024-09-10T18:52:21.618555+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:22.901 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:22 smithi151 bash[22565]: audit 2024-09-10T18:52:21.619298+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:23.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[18597]: audit 2024-09-10T18:52:21.575164+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]': finished 2024-09-10T18:52:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[18597]: cluster 2024-09-10T18:52:21.575335+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-09-10T18:52:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[18597]: audit 2024-09-10T18:52:21.618555+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:23.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[18597]: audit 2024-09-10T18:52:21.619298+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:23.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[31607]: audit 2024-09-10T18:52:21.575164+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]': finished 2024-09-10T18:52:23.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[31607]: cluster 2024-09-10T18:52:21.575335+0000 mon.a (mon.0) 3472 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-09-10T18:52:23.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[31607]: audit 2024-09-10T18:52:21.618555+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:23.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:22 smithi100 bash[31607]: audit 2024-09-10T18:52:21.619298+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-146"}]: dispatch 2024-09-10T18:52:23.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:52:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:52:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:23 smithi151 bash[22565]: cluster 2024-09-10T18:52:22.229552+0000 mgr.y (mgr.24512) 691 : cluster [DBG] pgmap v1183: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:52:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:23 smithi151 bash[22565]: cluster 2024-09-10T18:52:22.582447+0000 mon.a (mon.0) 3475 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:23 smithi151 bash[22565]: cluster 2024-09-10T18:52:22.582504+0000 mon.a (mon.0) 3476 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:52:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:23 smithi151 bash[22565]: cluster 2024-09-10T18:52:22.601841+0000 mon.a (mon.0) 3477 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-09-10T18:52:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:23 smithi151 bash[22565]: audit 2024-09-10T18:52:22.899595+0000 mgr.y (mgr.24512) 692 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:24.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[31607]: cluster 2024-09-10T18:52:22.229552+0000 mgr.y (mgr.24512) 691 : cluster [DBG] pgmap v1183: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[31607]: cluster 2024-09-10T18:52:22.582447+0000 mon.a (mon.0) 3475 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[31607]: cluster 2024-09-10T18:52:22.582504+0000 mon.a (mon.0) 3476 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[31607]: cluster 2024-09-10T18:52:22.601841+0000 mon.a (mon.0) 3477 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[31607]: audit 2024-09-10T18:52:22.899595+0000 mgr.y (mgr.24512) 692 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[18597]: cluster 2024-09-10T18:52:22.229552+0000 mgr.y (mgr.24512) 691 : cluster [DBG] pgmap v1183: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[18597]: cluster 2024-09-10T18:52:22.582447+0000 mon.a (mon.0) 3475 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[18597]: cluster 2024-09-10T18:52:22.582504+0000 mon.a (mon.0) 3476 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[18597]: cluster 2024-09-10T18:52:22.601841+0000 mon.a (mon.0) 3477 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-09-10T18:52:24.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:23 smithi100 bash[18597]: audit 2024-09-10T18:52:22.899595+0000 mgr.y (mgr.24512) 692 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:24.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:52:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:24.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:24 smithi151 bash[22565]: cluster 2024-09-10T18:52:23.612065+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-09-10T18:52:24.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:24 smithi151 bash[22565]: audit 2024-09-10T18:52:23.613573+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:24.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:24 smithi151 bash[22565]: audit 2024-09-10T18:52:24.343542+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:25.043 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:24 smithi100 bash[31607]: cluster 2024-09-10T18:52:23.612065+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-09-10T18:52:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:24 smithi100 bash[31607]: audit 2024-09-10T18:52:23.613573+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:25.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:24 smithi100 bash[31607]: audit 2024-09-10T18:52:24.343542+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:25.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:24 smithi100 bash[18597]: cluster 2024-09-10T18:52:23.612065+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-09-10T18:52:25.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:24 smithi100 bash[18597]: audit 2024-09-10T18:52:23.613573+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:25.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:24 smithi100 bash[18597]: audit 2024-09-10T18:52:24.343542+0000 mon.c (mon.2) 526 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:25.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:25 smithi151 bash[22565]: cluster 2024-09-10T18:52:24.230300+0000 mgr.y (mgr.24512) 693 : cluster [DBG] pgmap v1186: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:52:25.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:25 smithi151 bash[22565]: audit 2024-09-10T18:52:24.615436+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:25.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:25 smithi151 bash[22565]: cluster 2024-09-10T18:52:24.615662+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-09-10T18:52:25.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:25 smithi151 bash[22565]: audit 2024-09-10T18:52:24.643654+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:26.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[18597]: cluster 2024-09-10T18:52:24.230300+0000 mgr.y (mgr.24512) 693 : cluster [DBG] pgmap v1186: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:52:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[18597]: audit 2024-09-10T18:52:24.615436+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[18597]: cluster 2024-09-10T18:52:24.615662+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-09-10T18:52:26.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[18597]: audit 2024-09-10T18:52:24.643654+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[31607]: cluster 2024-09-10T18:52:24.230300+0000 mgr.y (mgr.24512) 693 : cluster [DBG] pgmap v1186: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:52:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[31607]: audit 2024-09-10T18:52:24.615436+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[31607]: cluster 2024-09-10T18:52:24.615662+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-09-10T18:52:26.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:25 smithi100 bash[31607]: audit 2024-09-10T18:52:24.643654+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:26.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:26 smithi151 bash[22565]: audit 2024-09-10T18:52:25.635979+0000 mon.a (mon.0) 3483 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:26.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:26 smithi151 bash[22565]: cluster 2024-09-10T18:52:25.636099+0000 mon.a (mon.0) 3484 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-09-10T18:52:26.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:26 smithi151 bash[22565]: audit 2024-09-10T18:52:25.652621+0000 mon.a (mon.0) 3485 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]: dispatch 2024-09-10T18:52:27.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:26 smithi100 bash[18597]: audit 2024-09-10T18:52:25.635979+0000 mon.a (mon.0) 3483 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:26 smithi100 bash[18597]: cluster 2024-09-10T18:52:25.636099+0000 mon.a (mon.0) 3484 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-09-10T18:52:27.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:26 smithi100 bash[18597]: audit 2024-09-10T18:52:25.652621+0000 mon.a (mon.0) 3485 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]: dispatch 2024-09-10T18:52:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:26 smithi100 bash[31607]: audit 2024-09-10T18:52:25.635979+0000 mon.a (mon.0) 3483 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:26 smithi100 bash[31607]: cluster 2024-09-10T18:52:25.636099+0000 mon.a (mon.0) 3484 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-09-10T18:52:27.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:26 smithi100 bash[31607]: audit 2024-09-10T18:52:25.652621+0000 mon.a (mon.0) 3485 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]: dispatch 2024-09-10T18:52:27.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:27 smithi151 bash[22565]: cluster 2024-09-10T18:52:26.231077+0000 mgr.y (mgr.24512) 694 : cluster [DBG] pgmap v1189: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:52:27.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:27 smithi151 bash[22565]: audit 2024-09-10T18:52:26.653650+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]': finished 2024-09-10T18:52:27.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:27 smithi151 bash[22565]: cluster 2024-09-10T18:52:26.653804+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-09-10T18:52:27.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:27 smithi151 bash[22565]: audit 2024-09-10T18:52:26.693578+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:27.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:27 smithi151 bash[22565]: audit 2024-09-10T18:52:26.694366+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]: dispatch 2024-09-10T18:52:28.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[18597]: cluster 2024-09-10T18:52:26.231077+0000 mgr.y (mgr.24512) 694 : cluster [DBG] pgmap v1189: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[18597]: audit 2024-09-10T18:52:26.653650+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]': finished 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[18597]: cluster 2024-09-10T18:52:26.653804+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[18597]: audit 2024-09-10T18:52:26.693578+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[18597]: audit 2024-09-10T18:52:26.694366+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]: dispatch 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[31607]: cluster 2024-09-10T18:52:26.231077+0000 mgr.y (mgr.24512) 694 : cluster [DBG] pgmap v1189: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[31607]: audit 2024-09-10T18:52:26.653650+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]': finished 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[31607]: cluster 2024-09-10T18:52:26.653804+0000 mon.a (mon.0) 3487 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[31607]: audit 2024-09-10T18:52:26.693578+0000 mon.a (mon.0) 3488 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:28.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:27 smithi100 bash[31607]: audit 2024-09-10T18:52:26.694366+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-148"}]: dispatch 2024-09-10T18:52:29.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:28 smithi100 bash[18597]: cluster 2024-09-10T18:52:27.681721+0000 mon.a (mon.0) 3490 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-09-10T18:52:29.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:28 smithi100 bash[31607]: cluster 2024-09-10T18:52:27.681721+0000 mon.a (mon.0) 3490 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-09-10T18:52:29.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:28 smithi151 bash[22565]: cluster 2024-09-10T18:52:27.681721+0000 mon.a (mon.0) 3490 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-09-10T18:52:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:29 smithi100 bash[18597]: cluster 2024-09-10T18:52:28.231817+0000 mgr.y (mgr.24512) 695 : cluster [DBG] pgmap v1192: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:29 smithi100 bash[18597]: cluster 2024-09-10T18:52:28.693238+0000 mon.a (mon.0) 3491 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-09-10T18:52:30.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:29 smithi100 bash[18597]: audit 2024-09-10T18:52:28.695020+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:30.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:29 smithi100 bash[31607]: cluster 2024-09-10T18:52:28.231817+0000 mgr.y (mgr.24512) 695 : cluster [DBG] pgmap v1192: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:30.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:29 smithi100 bash[31607]: cluster 2024-09-10T18:52:28.693238+0000 mon.a (mon.0) 3491 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-09-10T18:52:30.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:29 smithi100 bash[31607]: audit 2024-09-10T18:52:28.695020+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:30.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:29 smithi151 bash[22565]: cluster 2024-09-10T18:52:28.231817+0000 mgr.y (mgr.24512) 695 : cluster [DBG] pgmap v1192: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:30.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:29 smithi151 bash[22565]: cluster 2024-09-10T18:52:28.693238+0000 mon.a (mon.0) 3491 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-09-10T18:52:30.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:29 smithi151 bash[22565]: audit 2024-09-10T18:52:28.695020+0000 mon.a (mon.0) 3492 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:31.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:31 smithi151 bash[22565]: audit 2024-09-10T18:52:29.694902+0000 mon.a (mon.0) 3493 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:31.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:31 smithi151 bash[22565]: cluster 2024-09-10T18:52:29.695987+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-09-10T18:52:31.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:31 smithi151 bash[22565]: audit 2024-09-10T18:52:29.743934+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:31.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:31 smithi151 bash[22565]: audit 2024-09-10T18:52:29.744495+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-150"}]: dispatch 2024-09-10T18:52:31.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:31 smithi151 bash[22565]: cluster 2024-09-10T18:52:30.698684+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-09-10T18:52:31.543 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:31] "GET /metrics HTTP/1.1" 200 228682 "" "Prometheus/2.43.0" 2024-09-10T18:52:31.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[18597]: audit 2024-09-10T18:52:29.694902+0000 mon.a (mon.0) 3493 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:31.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[18597]: cluster 2024-09-10T18:52:29.695987+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-09-10T18:52:31.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[18597]: audit 2024-09-10T18:52:29.743934+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:31.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[18597]: audit 2024-09-10T18:52:29.744495+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-150"}]: dispatch 2024-09-10T18:52:31.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[18597]: cluster 2024-09-10T18:52:30.698684+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-09-10T18:52:31.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[31607]: audit 2024-09-10T18:52:29.694902+0000 mon.a (mon.0) 3493 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:31.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[31607]: cluster 2024-09-10T18:52:29.695987+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-09-10T18:52:31.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[31607]: audit 2024-09-10T18:52:29.743934+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:31.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[31607]: audit 2024-09-10T18:52:29.744495+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-150"}]: dispatch 2024-09-10T18:52:31.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:31 smithi100 bash[31607]: cluster 2024-09-10T18:52:30.698684+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-09-10T18:52:32.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:32 smithi151 bash[22565]: cluster 2024-09-10T18:52:30.232621+0000 mgr.y (mgr.24512) 696 : cluster [DBG] pgmap v1195: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:32.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:32 smithi151 bash[22565]: cluster 2024-09-10T18:52:31.159122+0000 mon.a (mon.0) 3498 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:32.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:32 smithi151 bash[22565]: cluster 2024-09-10T18:52:31.702819+0000 mon.a (mon.0) 3499 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-09-10T18:52:32.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:32 smithi151 bash[22565]: audit 2024-09-10T18:52:31.704247+0000 mon.a (mon.0) 3500 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:32.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[18597]: cluster 2024-09-10T18:52:30.232621+0000 mgr.y (mgr.24512) 696 : cluster [DBG] pgmap v1195: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[18597]: cluster 2024-09-10T18:52:31.159122+0000 mon.a (mon.0) 3498 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[18597]: cluster 2024-09-10T18:52:31.702819+0000 mon.a (mon.0) 3499 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-09-10T18:52:32.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[18597]: audit 2024-09-10T18:52:31.704247+0000 mon.a (mon.0) 3500 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[31607]: cluster 2024-09-10T18:52:30.232621+0000 mgr.y (mgr.24512) 696 : cluster [DBG] pgmap v1195: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[31607]: cluster 2024-09-10T18:52:31.159122+0000 mon.a (mon.0) 3498 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[31607]: cluster 2024-09-10T18:52:31.702819+0000 mon.a (mon.0) 3499 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-09-10T18:52:32.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:32 smithi100 bash[31607]: audit 2024-09-10T18:52:31.704247+0000 mon.a (mon.0) 3500 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:33.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:52:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:52:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:34 smithi151 bash[22565]: cluster 2024-09-10T18:52:32.233500+0000 mgr.y (mgr.24512) 697 : cluster [DBG] pgmap v1198: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:34 smithi151 bash[22565]: audit 2024-09-10T18:52:32.704537+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:34 smithi151 bash[22565]: cluster 2024-09-10T18:52:32.705289+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-09-10T18:52:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:34 smithi151 bash[22565]: audit 2024-09-10T18:52:32.769590+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:34 smithi151 bash[22565]: audit 2024-09-10T18:52:32.770459+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-152"}]: dispatch 2024-09-10T18:52:34.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:34 smithi151 bash[22565]: audit 2024-09-10T18:52:32.908326+0000 mgr.y (mgr.24512) 698 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[18597]: cluster 2024-09-10T18:52:32.233500+0000 mgr.y (mgr.24512) 697 : cluster [DBG] pgmap v1198: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[18597]: audit 2024-09-10T18:52:32.704537+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[18597]: cluster 2024-09-10T18:52:32.705289+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[18597]: audit 2024-09-10T18:52:32.769590+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[18597]: audit 2024-09-10T18:52:32.770459+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-152"}]: dispatch 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[18597]: audit 2024-09-10T18:52:32.908326+0000 mgr.y (mgr.24512) 698 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[31607]: cluster 2024-09-10T18:52:32.233500+0000 mgr.y (mgr.24512) 697 : cluster [DBG] pgmap v1198: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:34.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[31607]: audit 2024-09-10T18:52:32.704537+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:34.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[31607]: cluster 2024-09-10T18:52:32.705289+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-09-10T18:52:34.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[31607]: audit 2024-09-10T18:52:32.769590+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:34.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[31607]: audit 2024-09-10T18:52:32.770459+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-152"}]: dispatch 2024-09-10T18:52:34.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:34 smithi100 bash[31607]: audit 2024-09-10T18:52:32.908326+0000 mgr.y (mgr.24512) 698 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:34.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:52:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:35.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:35 smithi151 bash[22565]: cluster 2024-09-10T18:52:33.706996+0000 mon.a (mon.0) 3505 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-09-10T18:52:35.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:35 smithi151 bash[22565]: cluster 2024-09-10T18:52:34.715463+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-09-10T18:52:35.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:35 smithi151 bash[22565]: audit 2024-09-10T18:52:34.723349+0000 mon.a (mon.0) 3507 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:35.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:35 smithi100 bash[18597]: cluster 2024-09-10T18:52:33.706996+0000 mon.a (mon.0) 3505 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-09-10T18:52:35.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:35 smithi100 bash[18597]: cluster 2024-09-10T18:52:34.715463+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-09-10T18:52:35.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:35 smithi100 bash[18597]: audit 2024-09-10T18:52:34.723349+0000 mon.a (mon.0) 3507 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:35 smithi100 bash[31607]: cluster 2024-09-10T18:52:33.706996+0000 mon.a (mon.0) 3505 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-09-10T18:52:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:35 smithi100 bash[31607]: cluster 2024-09-10T18:52:34.715463+0000 mon.a (mon.0) 3506 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-09-10T18:52:35.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:35 smithi100 bash[31607]: audit 2024-09-10T18:52:34.723349+0000 mon.a (mon.0) 3507 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:36.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:36 smithi151 bash[22565]: cluster 2024-09-10T18:52:34.234243+0000 mgr.y (mgr.24512) 699 : cluster [DBG] pgmap v1201: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:36.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:36 smithi151 bash[22565]: audit 2024-09-10T18:52:35.716023+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:36.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:36 smithi151 bash[22565]: cluster 2024-09-10T18:52:35.716232+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-09-10T18:52:36.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:36 smithi151 bash[22565]: audit 2024-09-10T18:52:35.719326+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-111","var": "dedup_tier","val": "test-rados-api-smithi100-72764-154"}]: dispatch 2024-09-10T18:52:36.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:36 smithi151 bash[22565]: audit 2024-09-10T18:52:35.748634+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:36.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:36 smithi151 bash[22565]: audit 2024-09-10T18:52:35.749037+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-154"}]: dispatch 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[18597]: cluster 2024-09-10T18:52:34.234243+0000 mgr.y (mgr.24512) 699 : cluster [DBG] pgmap v1201: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[18597]: audit 2024-09-10T18:52:35.716023+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[18597]: cluster 2024-09-10T18:52:35.716232+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[18597]: audit 2024-09-10T18:52:35.719326+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-111","var": "dedup_tier","val": "test-rados-api-smithi100-72764-154"}]: dispatch 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[18597]: audit 2024-09-10T18:52:35.748634+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[18597]: audit 2024-09-10T18:52:35.749037+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-154"}]: dispatch 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[31607]: cluster 2024-09-10T18:52:34.234243+0000 mgr.y (mgr.24512) 699 : cluster [DBG] pgmap v1201: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:36.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[31607]: audit 2024-09-10T18:52:35.716023+0000 mon.a (mon.0) 3508 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-72764-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[31607]: cluster 2024-09-10T18:52:35.716232+0000 mon.a (mon.0) 3509 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-09-10T18:52:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[31607]: audit 2024-09-10T18:52:35.719326+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-72764-111","var": "dedup_tier","val": "test-rados-api-smithi100-72764-154"}]: dispatch 2024-09-10T18:52:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[31607]: audit 2024-09-10T18:52:35.748634+0000 mon.a (mon.0) 3511 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:36.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:36 smithi100 bash[31607]: audit 2024-09-10T18:52:35.749037+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-72764-111", "tierpool": "test-rados-api-smithi100-72764-154"}]: dispatch 2024-09-10T18:52:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:37 smithi100 bash[31607]: cluster 2024-09-10T18:52:36.235046+0000 mgr.y (mgr.24512) 700 : cluster [DBG] pgmap v1204: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T18:52:38.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:37 smithi100 bash[31607]: cluster 2024-09-10T18:52:36.722798+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-09-10T18:52:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:37 smithi100 bash[18597]: cluster 2024-09-10T18:52:36.235046+0000 mgr.y (mgr.24512) 700 : cluster [DBG] pgmap v1204: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T18:52:38.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:37 smithi100 bash[18597]: cluster 2024-09-10T18:52:36.722798+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-09-10T18:52:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:37 smithi151 bash[22565]: cluster 2024-09-10T18:52:36.235046+0000 mgr.y (mgr.24512) 700 : cluster [DBG] pgmap v1204: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T18:52:38.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:37 smithi151 bash[22565]: cluster 2024-09-10T18:52:36.722798+0000 mon.a (mon.0) 3513 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-09-10T18:52:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:38 smithi100 bash[31607]: cluster 2024-09-10T18:52:37.722911+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-09-10T18:52:39.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:38 smithi100 bash[31607]: audit 2024-09-10T18:52:37.725685+0000 mon.a (mon.0) 3515 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:39.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:38 smithi100 bash[18597]: cluster 2024-09-10T18:52:37.722911+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-09-10T18:52:39.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:38 smithi100 bash[18597]: audit 2024-09-10T18:52:37.725685+0000 mon.a (mon.0) 3515 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:39.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:38 smithi151 bash[22565]: cluster 2024-09-10T18:52:37.722911+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-09-10T18:52:39.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:38 smithi151 bash[22565]: audit 2024-09-10T18:52:37.725685+0000 mon.a (mon.0) 3515 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:39.741 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7807 ms) 2024-09-10T18:52:39.741 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-10T18:52:39.741 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13089 ms) 2024-09-10T18:52:39.741 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8089 ms) 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13127 ms) 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7715 ms) 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8091 ms) 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: ok, hit_set contains 255:602f83fe:::foo:head 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9046 ms) 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,0 2024-09-10T18:52:39.742 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first is 1725994275 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,1725994280,1725994281,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,1725994280,1725994281,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,1725994280,1725994281,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,1725994280,1725994281,1725994283,1725994284,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,1725994280,1725994281,1725994283,1725994284,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994275,1725994277,1725994278,1725994280,1725994281,1725994283,1725994284,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1725994278,1725994280,1725994281,1725994283,1725994284,1725994286,1725994287,0 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first now 1725994278, trimmed 2024-09-10T18:52:39.743 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21001 ms) 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: foo0 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14082 ms) 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18013 ms) 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22220 ms) 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5070 ms) 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3020 ms) 2024-09-10T18:52:39.744 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3007 ms) 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3017 ms) 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (231823 ms total) 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (865362 ms total) 2024-09-10T18:52:39.745 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-10T18:52:39.747 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72883 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72883 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72636 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72636 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72619 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72619 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72689 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72689 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.748 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72940 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72940 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73044 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 73044 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72782 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72782 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72596 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72596 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73115 2024-09-10T18:52:39.749 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 73115 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72983 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72983 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72824 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72824 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=73076 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 73076 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72628 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72628 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72991 2024-09-10T18:52:39.750 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72991 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72612 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72612 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72959 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72959 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72714 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72714 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72975 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72975 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=72794 2024-09-10T18:52:39.751 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 72794 2024-09-10T18:52:39.752 INFO:tasks.workunit.client.0.smithi100.stderr:+ exit 0 2024-09-10T18:52:39.752 INFO:tasks.workunit.client.0.smithi100.stderr:+ cleanup 2024-09-10T18:52:39.752 INFO:tasks.workunit.client.0.smithi100.stderr:+ pkill -P 72582 2024-09-10T18:52:39.756 INFO:tasks.workunit.client.0.smithi100.stderr:+ true 2024-09-10T18:52:39.757 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T18:52:39.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T18:52:39.770 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-10T18:52:39.771 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-10T18:52:39.771 DEBUG:teuthology.orchestra.run.smithi100:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fad5356a94e6044ce1062d748daa5a0f2ed53a7d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-10T18:52:39.837 INFO:tasks.workunit.client.0.smithi100.stderr:+ uuidgen 2024-09-10T18:52:39.838 INFO:tasks.workunit.client.0.smithi100.stderr:+ p=3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:52:39.838 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool create 3713687d-818c-43e2-97c0-4c4d90a80f33 12 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- 172.21.15.100:0/2442124312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 msgr2=0x7f7710109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 --2- 172.21.15.100:0/2442124312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 0x7f7710109240 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f77000049c0 tx=0x7f7700009e30 comp rx=0 tx=0).stop 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- 172.21.15.100:0/2442124312 shutdown_connections 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 --2- 172.21.15.100:0/2442124312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7710109780 0x7f771010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 --2- 172.21.15.100:0/2442124312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 0x7f7710109240 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 --2- 172.21.15.100:0/2442124312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77100fbbd0 0x7f77100fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- 172.21.15.100:0/2442124312 >> 172.21.15.100:0/2442124312 conn(0x7f7710071930 msgr2=0x7f77100f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:39.951 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- 172.21.15.100:0/2442124312 shutdown_connections 2024-09-10T18:52:39.952 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- 172.21.15.100:0/2442124312 wait complete. 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 Processor -- start 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- start start 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77100fbbd0 0x7f7710115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 0x7f7710115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7710109780 0x7f771010f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771010fbb0 con 0x7f77100fc4e0 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771010fd20 con 0x7f7710109780 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f7715346700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771010fee0 con 0x7f77100fbbd0 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f770effd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77100fbbd0 0x7f7710115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f770e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 0x7f7710115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:39.954 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f770effd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77100fbbd0 0x7f7710115410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:60814/0 (socket says 172.21.15.100:60814) 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.949+0000 7f770effd700 1 -- 172.21.15.100:0/2261366064 learned_addr learned my addr 172.21.15.100:0/2261366064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f770f7fe700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7710109780 0x7f771010f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f770effd700 1 -- 172.21.15.100:0/2261366064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7710109780 msgr2=0x7f771010f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f770effd700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7710109780 0x7f771010f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f770effd700 1 -- 172.21.15.100:0/2261366064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 msgr2=0x7f7710115950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f770effd700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 0x7f7710115950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f770effd700 1 -- 172.21.15.100:0/2261366064 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77080048f0 con 0x7f77100fbbd0 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f770effd700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77100fbbd0 0x7f7710115410 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f770400ebc0 tx=0x7f770400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704007cc0 con 0x7f77100fbbd0 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f770400f040 con 0x7f77100fbbd0 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7700005040 con 0x7f77100fbbd0 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704009340 con 0x7f77100fbbd0 2024-09-10T18:52:39.955 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771011c2a0 con 0x7f77100fbbd0 2024-09-10T18:52:39.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7710066b30 con 0x7f77100fbbd0 2024-09-10T18:52:39.956 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.953+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f7704009770 con 0x7f77100fbbd0 2024-09-10T18:52:39.957 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.957+0000 7f76f77fe700 1 --2- 172.21.15.100:0/2261366064 >> v2:172.21.15.100:6800/2155730476 conn(0x7f76f806f010 0x7f76f80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:39.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.957+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(746..746 src has 1..746) v4 ==== 7399+0+0 (secure 0 0 0) 0x7f770408e5a0 con 0x7f77100fbbd0 2024-09-10T18:52:39.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.957+0000 7f770e7fc700 1 --2- 172.21.15.100:0/2261366064 >> v2:172.21.15.100:6800/2155730476 conn(0x7f76f806f010 0x7f76f80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:39.960 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.957+0000 7f770e7fc700 1 --2- 172.21.15.100:0/2261366064 >> v2:172.21.15.100:6800/2155730476 conn(0x7f76f806f010 0x7f76f80714c0 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7f7700009b80 tx=0x7f7700009e30 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:39.964 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:39.961+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f770404e2a0 con 0x7f77100fbbd0 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: cluster 2024-09-10T18:52:38.235855+0000 mgr.y (mgr.24512) 701 : cluster [DBG] pgmap v1207: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: audit 2024-09-10T18:52:38.725302+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: cluster 2024-09-10T18:52:38.725810+0000 mon.a (mon.0) 3517 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: cluster 2024-09-10T18:52:38.727923+0000 mon.a (mon.0) 3518 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: audit 2024-09-10T18:52:38.728847+0000 mon.a (mon.0) 3519 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: audit 2024-09-10T18:52:39.343733+0000 mon.c (mon.2) 527 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: audit 2024-09-10T18:52:39.732822+0000 mon.a (mon.0) 3520 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[31607]: cluster 2024-09-10T18:52:39.732953+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: cluster 2024-09-10T18:52:38.235855+0000 mgr.y (mgr.24512) 701 : cluster [DBG] pgmap v1207: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:40.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: audit 2024-09-10T18:52:38.725302+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:40.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: cluster 2024-09-10T18:52:38.725810+0000 mon.a (mon.0) 3517 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-09-10T18:52:40.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: cluster 2024-09-10T18:52:38.727923+0000 mon.a (mon.0) 3518 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:40.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: audit 2024-09-10T18:52:38.728847+0000 mon.a (mon.0) 3519 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:40.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: audit 2024-09-10T18:52:39.343733+0000 mon.c (mon.2) 527 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:40.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: audit 2024-09-10T18:52:39.732822+0000 mon.a (mon.0) 3520 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:40.045 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:39 smithi100 bash[18597]: cluster 2024-09-10T18:52:39.732953+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-09-10T18:52:40.138 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.137+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12} v 0) v1 -- 0x7f771010e470 con 0x7f77100fbbd0 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: cluster 2024-09-10T18:52:38.235855+0000 mgr.y (mgr.24512) 701 : cluster [DBG] pgmap v1207: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: audit 2024-09-10T18:52:38.725302+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: cluster 2024-09-10T18:52:38.725810+0000 mon.a (mon.0) 3517 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: cluster 2024-09-10T18:52:38.727923+0000 mon.a (mon.0) 3518 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: audit 2024-09-10T18:52:38.728847+0000 mon.a (mon.0) 3519 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]: dispatch 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: audit 2024-09-10T18:52:39.343733+0000 mon.c (mon.2) 527 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: audit 2024-09-10T18:52:39.732822+0000 mon.a (mon.0) 3520 : audit [INF] from='client.? 172.21.15.100:0/131602753' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-72764-111"}]': finished 2024-09-10T18:52:40.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:39 smithi151 bash[22565]: cluster 2024-09-10T18:52:39.732953+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-09-10T18:52:40.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.757+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]=0 pool '3713687d-818c-43e2-97c0-4c4d90a80f33' created v747) v1 ==== 176+0+0 (secure 0 0 0) 0x7f77040190c0 con 0x7f77100fbbd0 2024-09-10T18:52:40.866 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.865+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12} v 0) v1 -- 0x7f771004c430 con 0x7f77100fbbd0 2024-09-10T18:52:40.867 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.865+0000 7f76f77fe700 1 -- 172.21.15.100:0/2261366064 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]=0 pool '3713687d-818c-43e2-97c0-4c4d90a80f33' already exists v747) v1 ==== 183+0+0 (secure 0 0 0) 0x7f770405af40 con 0x7f77100fbbd0 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 >> v2:172.21.15.100:6800/2155730476 conn(0x7f76f806f010 msgr2=0x7f76f80714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 --2- 172.21.15.100:0/2261366064 >> v2:172.21.15.100:6800/2155730476 conn(0x7f76f806f010 0x7f76f80714c0 secure :-1 s=READY pgs=3516 cs=0 l=1 rev1=1 crypto rx=0x7f7700009b80 tx=0x7f7700009e30 comp rx=0 tx=0).stop 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77100fbbd0 msgr2=0x7f7710115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77100fbbd0 0x7f7710115410 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f770400ebc0 tx=0x7f770400b040 comp rx=0 tx=0).stop 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 shutdown_connections 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 --2- 172.21.15.100:0/2261366064 >> v2:172.21.15.100:6800/2155730476 conn(0x7f76f806f010 0x7f76f80714c0 unknown :-1 s=CLOSED pgs=3516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7710109780 0x7f771010f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.870 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77100fc4e0 0x7f7710115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 --2- 172.21.15.100:0/2261366064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77100fbbd0 0x7f7710115410 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 >> 172.21.15.100:0/2261366064 conn(0x7f7710071930 msgr2=0x7f7710071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:40.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 shutdown_connections 2024-09-10T18:52:40.871 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.869+0000 7f7715346700 1 -- 172.21.15.100:0/2261366064 wait complete. 2024-09-10T18:52:40.871 INFO:tasks.workunit.client.0.smithi100.stderr:pool '3713687d-818c-43e2-97c0-4c4d90a80f33' already exists 2024-09-10T18:52:40.879 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 3713687d-818c-43e2-97c0-4c4d90a80f33 max_objects 10 2024-09-10T18:52:40.982 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 -- 172.21.15.100:0/2350389598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 msgr2=0x7f19a010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:40.982 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/2350389598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010bae0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f1998009d60 tx=0x7f1998004d10 comp rx=0 tx=0).stop 2024-09-10T18:52:40.982 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 -- 172.21.15.100:0/2350389598 shutdown_connections 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/2350389598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010bae0 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/2350389598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a00fc460 0x7f19a01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/2350389598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a00fbb50 0x7f19a00fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 -- 172.21.15.100:0/2350389598 >> 172.21.15.100:0/2350389598 conn(0x7f19a0071930 msgr2=0x7f19a00f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 -- 172.21.15.100:0/2350389598 shutdown_connections 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 -- 172.21.15.100:0/2350389598 wait complete. 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 Processor -- start 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.977+0000 7f19a4bc7700 1 -- start start 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a00fbb50 0x7f19a01151a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a00fc460 0x7f19a01156e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:40.983 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a010f990 con 0x7f19a00fc460 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a010fb00 con 0x7f19a00fbb50 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a010fcc0 con 0x7f19a0109700 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:52604/0 (socket says 172.21.15.100:52604) 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 -- 172.21.15.100:0/362894846 learned_addr learned my addr 172.21.15.100:0/362894846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199dd9b700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a00fc460 0x7f19a01156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199e59c700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a00fbb50 0x7f19a01151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 -- 172.21.15.100:0/362894846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a00fbb50 msgr2=0x7f19a01151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a00fbb50 0x7f19a01151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 -- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a00fc460 msgr2=0x7f19a01156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:40.984 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a00fc460 0x7f19a01156e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 -- 172.21.15.100:0/362894846 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998009940 con 0x7f19a0109700 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199dd9b700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a00fc460 0x7f19a01156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f199ed9d700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010f340 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f1998008dd0 tx=0x7f1998004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199800e030 con 0x7f19a0109700 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a010ff40 con 0x7f19a0109700 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f19980097d0 con 0x7f19a0109700 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998012680 con 0x7f19a0109700 2024-09-10T18:52:40.985 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a011c190 con 0x7f19a0109700 2024-09-10T18:52:40.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a01038d0 con 0x7f19a0109700 2024-09-10T18:52:40.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f1998012ab0 con 0x7f19a0109700 2024-09-10T18:52:40.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f198effd700 1 --2- 172.21.15.100:0/362894846 >> v2:172.21.15.100:6800/2155730476 conn(0x7f198806f010 0x7f19880714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:40.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.981+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(747..747 src has 1..747) v4 ==== 7774+0+0 (secure 0 0 0) 0x7f1998090ca0 con 0x7f19a0109700 2024-09-10T18:52:40.986 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.985+0000 7f199e59c700 1 --2- 172.21.15.100:0/362894846 >> v2:172.21.15.100:6800/2155730476 conn(0x7f198806f010 0x7f19880714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:40.987 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.985+0000 7f199e59c700 1 --2- 172.21.15.100:0/362894846 >> v2:172.21.15.100:6800/2155730476 conn(0x7f198806f010 0x7f19880714c0 secure :-1 s=READY pgs=3517 cs=0 l=1 rev1=1 crypto rx=0x7f1994004600 tx=0x7f199400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:40.992 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:40.989+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f199805d4c0 con 0x7f19a0109700 2024-09-10T18:52:41.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:40 smithi100 bash[18597]: audit 2024-09-10T18:52:40.139549+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.100:0/2261366064' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:41.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:40 smithi100 bash[18597]: audit 2024-09-10T18:52:40.140247+0000 mon.a (mon.0) 3522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:40 smithi100 bash[31607]: audit 2024-09-10T18:52:40.139549+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.100:0/2261366064' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:41.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:40 smithi100 bash[31607]: audit 2024-09-10T18:52:40.140247+0000 mon.a (mon.0) 3522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:41.155 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:41.153+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f19a004c430 con 0x7f19a0109700 2024-09-10T18:52:41.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:40 smithi151 bash[22565]: audit 2024-09-10T18:52:40.139549+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.100:0/2261366064' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:41.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:40 smithi151 bash[22565]: audit 2024-09-10T18:52:40.140247+0000 mon.a (mon.0) 3522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:41.766 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:41.765+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v748) v1 ==== 223+0+0 (secure 0 0 0) 0x7f19980200d0 con 0x7f19a0109700 2024-09-10T18:52:41.772 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:41] "GET /metrics HTTP/1.1" 200 223867 "" "Prometheus/2.43.0" 2024-09-10T18:52:41.873 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:41.873+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f19a010e410 con 0x7f19a0109700 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18597]: cluster 2024-09-10T18:52:40.236509+0000 mgr.y (mgr.24512) 702 : cluster [DBG] pgmap v1210: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18597]: audit 2024-09-10T18:52:40.756288+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]': finished 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18597]: cluster 2024-09-10T18:52:40.756370+0000 mon.a (mon.0) 3524 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18597]: audit 2024-09-10T18:52:40.867225+0000 mon.c (mon.2) 529 : audit [INF] from='client.? 172.21.15.100:0/2261366064' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18597]: audit 2024-09-10T18:52:40.867806+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18597]: audit 2024-09-10T18:52:41.156539+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.100:0/362894846' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[18597]: audit 2024-09-10T18:52:41.157202+0000 mon.a (mon.0) 3526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:42.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[31607]: cluster 2024-09-10T18:52:40.236509+0000 mgr.y (mgr.24512) 702 : cluster [DBG] pgmap v1210: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:42.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[31607]: audit 2024-09-10T18:52:40.756288+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]': finished 2024-09-10T18:52:42.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[31607]: cluster 2024-09-10T18:52:40.756370+0000 mon.a (mon.0) 3524 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-09-10T18:52:42.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[31607]: audit 2024-09-10T18:52:40.867225+0000 mon.c (mon.2) 529 : audit [INF] from='client.? 172.21.15.100:0/2261366064' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:42.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[31607]: audit 2024-09-10T18:52:40.867806+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:42.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[31607]: audit 2024-09-10T18:52:41.156539+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.100:0/362894846' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:42.045 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:41 smithi100 bash[31607]: audit 2024-09-10T18:52:41.157202+0000 mon.a (mon.0) 3526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:41 smithi151 bash[22565]: cluster 2024-09-10T18:52:40.236509+0000 mgr.y (mgr.24512) 702 : cluster [DBG] pgmap v1210: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:41 smithi151 bash[22565]: audit 2024-09-10T18:52:40.756288+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]': finished 2024-09-10T18:52:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:41 smithi151 bash[22565]: cluster 2024-09-10T18:52:40.756370+0000 mon.a (mon.0) 3524 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-09-10T18:52:42.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:41 smithi151 bash[22565]: audit 2024-09-10T18:52:40.867225+0000 mon.c (mon.2) 529 : audit [INF] from='client.? 172.21.15.100:0/2261366064' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:42.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:41 smithi151 bash[22565]: audit 2024-09-10T18:52:40.867806+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pg_num": 12}]: dispatch 2024-09-10T18:52:42.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:41 smithi151 bash[22565]: audit 2024-09-10T18:52:41.156539+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.100:0/362894846' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:42.173 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:41 smithi151 bash[22565]: audit 2024-09-10T18:52:41.157202+0000 mon.a (mon.0) 3526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:42.797 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f198effd700 1 -- 172.21.15.100:0/362894846 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v749) v1 ==== 223+0+0 (secure 0 0 0) 0x7f1998017070 con 0x7f19a0109700 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 >> v2:172.21.15.100:6800/2155730476 conn(0x7f198806f010 msgr2=0x7f19880714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/362894846 >> v2:172.21.15.100:6800/2155730476 conn(0x7f198806f010 0x7f19880714c0 secure :-1 s=READY pgs=3517 cs=0 l=1 rev1=1 crypto rx=0x7f1994004600 tx=0x7f199400b040 comp rx=0 tx=0).stop 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 msgr2=0x7f19a010f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010f340 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f1998008dd0 tx=0x7f1998004d10 comp rx=0 tx=0).stop 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 shutdown_connections 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/362894846 >> v2:172.21.15.100:6800/2155730476 conn(0x7f198806f010 0x7f19880714c0 unknown :-1 s=CLOSED pgs=3517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19a0109700 0x7f19a010f340 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a00fc460 0x7f19a01156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 --2- 172.21.15.100:0/362894846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19a00fbb50 0x7f19a01151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 >> 172.21.15.100:0/362894846 conn(0x7f19a0071930 msgr2=0x7f19a0073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:42.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 shutdown_connections 2024-09-10T18:52:42.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.797+0000 7f19a4bc7700 1 -- 172.21.15.100:0/362894846 wait complete. 2024-09-10T18:52:42.801 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 10 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:52:42.810 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool application enable 3713687d-818c-43e2-97c0-4c4d90a80f33 rados 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- 172.21.15.100:0/2076063480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01240fddf0 msgr2=0x7f01240fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 --2- 172.21.15.100:0/2076063480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01240fddf0 0x7f01240fe1c0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f01180049c0 tx=0x7f0118009e30 comp rx=0 tx=0).stop 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- 172.21.15.100:0/2076063480 shutdown_connections 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 --2- 172.21.15.100:0/2076063480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01241073a0 0x7f0124109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 --2- 172.21.15.100:0/2076063480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fe790 0x7f0124106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 --2- 172.21.15.100:0/2076063480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01240fddf0 0x7f01240fe1c0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- 172.21.15.100:0/2076063480 >> 172.21.15.100:0/2076063480 conn(0x7f0124071930 msgr2=0x7f01240f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- 172.21.15.100:0/2076063480 shutdown_connections 2024-09-10T18:52:42.915 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- 172.21.15.100:0/2076063480 wait complete. 2024-09-10T18:52:42.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 Processor -- start 2024-09-10T18:52:42.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- start start 2024-09-10T18:52:42.917 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fddf0 0x7f0124113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01240fe790 0x7f0124113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01241073a0 0x7f012410d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012410d9e0 con 0x7f01241073a0 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012410db50 con 0x7f01240fe790 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f012a137700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012410dd10 con 0x7f01240fddf0 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f01237fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fddf0 0x7f0124113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f0123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01241073a0 0x7f012410d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f0123fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01241073a0 0x7f012410d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46134/0 (socket says 172.21.15.100:46134) 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f01237fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fddf0 0x7f0124113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:52626/0 (socket says 172.21.15.100:52626) 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f0123fff700 1 -- 172.21.15.100:0/3083627511 learned_addr learned my addr 172.21.15.100:0/3083627511 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:52:42.918 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.913+0000 7f0122ffd700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01240fe790 0x7f0124113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f0123fff700 1 -- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fddf0 msgr2=0x7f0124113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f0123fff700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fddf0 0x7f0124113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f0123fff700 1 -- 172.21.15.100:0/3083627511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01240fe790 msgr2=0x7f0124113740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f0123fff700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01240fe790 0x7f0124113740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f0123fff700 1 -- 172.21.15.100:0/3083627511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0114005040 con 0x7f01241073a0 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f01237fe700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fddf0 0x7f0124113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f0123fff700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01241073a0 0x7f012410d390 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f011c001cd0 tx=0x7f011c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c004020 con 0x7f01241073a0 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f011c00f040 con 0x7f01241073a0 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0118005040 con 0x7f01241073a0 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0097b0 con 0x7f01241073a0 2024-09-10T18:52:42.919 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012411a050 con 0x7f01241073a0 2024-09-10T18:52:42.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0124101b90 con 0x7f01241073a0 2024-09-10T18:52:42.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.917+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f011c007b30 con 0x7f01241073a0 2024-09-10T18:52:42.920 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.921+0000 7f010bfff700 1 --2- 172.21.15.100:0/3083627511 >> v2:172.21.15.100:6800/2155730476 conn(0x7f010c06f010 0x7f010c0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:42.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.921+0000 7f01237fe700 1 --2- 172.21.15.100:0/3083627511 >> v2:172.21.15.100:6800/2155730476 conn(0x7f010c06f010 0x7f010c0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:42.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.921+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(749..749 src has 1..749) v4 ==== 7774+0+0 (secure 0 0 0) 0x7f011c08e600 con 0x7f01241073a0 2024-09-10T18:52:42.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.921+0000 7f01237fe700 1 --2- 172.21.15.100:0/3083627511 >> v2:172.21.15.100:6800/2155730476 conn(0x7f010c06f010 0x7f010c0714c0 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7f0118007f50 tx=0x7f0118009e30 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:42.933 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:42.933+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f011c05aea0 con 0x7f01241073a0 2024-09-10T18:52:43.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[18597]: audit 2024-09-10T18:52:41.765186+0000 mon.a (mon.0) 3527 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:52:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[18597]: cluster 2024-09-10T18:52:41.765313+0000 mon.a (mon.0) 3528 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-09-10T18:52:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[18597]: audit 2024-09-10T18:52:41.874705+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.100:0/362894846' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:43.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[18597]: audit 2024-09-10T18:52:41.875560+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[31607]: audit 2024-09-10T18:52:41.765186+0000 mon.a (mon.0) 3527 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:52:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[31607]: cluster 2024-09-10T18:52:41.765313+0000 mon.a (mon.0) 3528 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-09-10T18:52:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[31607]: audit 2024-09-10T18:52:41.874705+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.100:0/362894846' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:43.044 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:42 smithi100 bash[31607]: audit 2024-09-10T18:52:41.875560+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:43.108 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:43.105+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"} v 0) v1 -- 0x7f012410c260 con 0x7f01241073a0 2024-09-10T18:52:43.172 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:52:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:52:43.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:42 smithi151 bash[22565]: audit 2024-09-10T18:52:41.765186+0000 mon.a (mon.0) 3527 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:52:43.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:42 smithi151 bash[22565]: cluster 2024-09-10T18:52:41.765313+0000 mon.a (mon.0) 3528 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-09-10T18:52:43.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:42 smithi151 bash[22565]: audit 2024-09-10T18:52:41.874705+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.100:0/362894846' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:43.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:42 smithi151 bash[22565]: audit 2024-09-10T18:52:41.875560+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:52:43.794 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:43.793+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]=0 enabled application 'rados' on pool '3713687d-818c-43e2-97c0-4c4d90a80f33' v750) v1 ==== 213+0+0 (secure 0 0 0) 0x7f011c05a840 con 0x7f01241073a0 2024-09-10T18:52:43.902 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:43.901+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"} v 0) v1 -- 0x7f012410e870 con 0x7f01241073a0 2024-09-10T18:52:44.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:43 smithi151 bash[22565]: cluster 2024-09-10T18:52:42.237284+0000 mgr.y (mgr.24512) 703 : cluster [DBG] pgmap v1213: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:44.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:43 smithi151 bash[22565]: audit 2024-09-10T18:52:42.792774+0000 mon.a (mon.0) 3530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:52:44.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:43 smithi151 bash[22565]: cluster 2024-09-10T18:52:42.793771+0000 mon.a (mon.0) 3531 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-09-10T18:52:44.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:43 smithi151 bash[22565]: audit 2024-09-10T18:52:42.916189+0000 mgr.y (mgr.24512) 704 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:44.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:43 smithi151 bash[22565]: audit 2024-09-10T18:52:43.109162+0000 mon.a (mon.0) 3532 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]: dispatch 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[18597]: cluster 2024-09-10T18:52:42.237284+0000 mgr.y (mgr.24512) 703 : cluster [DBG] pgmap v1213: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[18597]: audit 2024-09-10T18:52:42.792774+0000 mon.a (mon.0) 3530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[18597]: cluster 2024-09-10T18:52:42.793771+0000 mon.a (mon.0) 3531 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[18597]: audit 2024-09-10T18:52:42.916189+0000 mgr.y (mgr.24512) 704 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[18597]: audit 2024-09-10T18:52:43.109162+0000 mon.a (mon.0) 3532 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]: dispatch 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[31607]: cluster 2024-09-10T18:52:42.237284+0000 mgr.y (mgr.24512) 703 : cluster [DBG] pgmap v1213: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[31607]: audit 2024-09-10T18:52:42.792774+0000 mon.a (mon.0) 3530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:52:44.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[31607]: cluster 2024-09-10T18:52:42.793771+0000 mon.a (mon.0) 3531 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-09-10T18:52:44.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[31607]: audit 2024-09-10T18:52:42.916189+0000 mgr.y (mgr.24512) 704 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:44.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:43 smithi100 bash[31607]: audit 2024-09-10T18:52:43.109162+0000 mon.a (mon.0) 3532 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]: dispatch 2024-09-10T18:52:44.803 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:52:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:44.812 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.809+0000 7f010bfff700 1 -- 172.21.15.100:0/3083627511 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]=0 enabled application 'rados' on pool '3713687d-818c-43e2-97c0-4c4d90a80f33' v751) v1 ==== 213+0+0 (secure 0 0 0) 0x7f011c0170d0 con 0x7f01241073a0 2024-09-10T18:52:44.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.813+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 >> v2:172.21.15.100:6800/2155730476 conn(0x7f010c06f010 msgr2=0x7f010c0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:44.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.813+0000 7f012a137700 1 --2- 172.21.15.100:0/3083627511 >> v2:172.21.15.100:6800/2155730476 conn(0x7f010c06f010 0x7f010c0714c0 secure :-1 s=READY pgs=3518 cs=0 l=1 rev1=1 crypto rx=0x7f0118007f50 tx=0x7f0118009e30 comp rx=0 tx=0).stop 2024-09-10T18:52:44.816 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.813+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01241073a0 msgr2=0x7f012410d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.813+0000 7f012a137700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01241073a0 0x7f012410d390 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f011c001cd0 tx=0x7f011c009ef0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 shutdown_connections 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 --2- 172.21.15.100:0/3083627511 >> v2:172.21.15.100:6800/2155730476 conn(0x7f010c06f010 0x7f010c0714c0 unknown :-1 s=CLOSED pgs=3518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01241073a0 0x7f012410d390 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01240fe790 0x7f0124113740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 --2- 172.21.15.100:0/3083627511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01240fddf0 0x7f0124113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 >> 172.21.15.100:0/3083627511 conn(0x7f0124071930 msgr2=0x7f01240fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:44.817 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 shutdown_connections 2024-09-10T18:52:44.818 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:52:44.817+0000 7f012a137700 1 -- 172.21.15.100:0/3083627511 wait complete. 2024-09-10T18:52:44.818 INFO:tasks.workunit.client.0.smithi100.stderr:enabled application 'rados' on pool '3713687d-818c-43e2-97c0-4c4d90a80f33' 2024-09-10T18:52:44.828 INFO:tasks.workunit.client.0.smithi100.stderr:+ seq 1 10 2024-09-10T18:52:44.840 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj1 /etc/passwd 2024-09-10T18:52:44.893 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj2 /etc/passwd 2024-09-10T18:52:44.949 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj3 /etc/passwd 2024-09-10T18:52:44.994 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj4 /etc/passwd 2024-09-10T18:52:45.041 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj5 /etc/passwd 2024-09-10T18:52:45.086 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj6 /etc/passwd 2024-09-10T18:52:45.133 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj7 /etc/passwd 2024-09-10T18:52:45.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:44 smithi151 bash[22565]: audit 2024-09-10T18:52:43.794522+0000 mon.a (mon.0) 3533 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]': finished 2024-09-10T18:52:45.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:44 smithi151 bash[22565]: cluster 2024-09-10T18:52:43.794747+0000 mon.a (mon.0) 3534 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-09-10T18:52:45.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:44 smithi151 bash[22565]: audit 2024-09-10T18:52:43.902882+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]: dispatch 2024-09-10T18:52:45.175 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj8 /etc/passwd 2024-09-10T18:52:45.221 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj9 /etc/passwd 2024-09-10T18:52:45.262 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put obj10 /etc/passwd 2024-09-10T18:52:45.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:44 smithi100 bash[31607]: audit 2024-09-10T18:52:43.794522+0000 mon.a (mon.0) 3533 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]': finished 2024-09-10T18:52:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:44 smithi100 bash[31607]: cluster 2024-09-10T18:52:43.794747+0000 mon.a (mon.0) 3534 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-09-10T18:52:45.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:44 smithi100 bash[31607]: audit 2024-09-10T18:52:43.902882+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]: dispatch 2024-09-10T18:52:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:44 smithi100 bash[18597]: audit 2024-09-10T18:52:43.794522+0000 mon.a (mon.0) 3533 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]': finished 2024-09-10T18:52:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:44 smithi100 bash[18597]: cluster 2024-09-10T18:52:43.794747+0000 mon.a (mon.0) 3534 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-09-10T18:52:45.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:44 smithi100 bash[18597]: audit 2024-09-10T18:52:43.902882+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]: dispatch 2024-09-10T18:52:45.310 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-09-10T18:52:46.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:45 smithi151 bash[22565]: cluster 2024-09-10T18:52:44.238082+0000 mgr.y (mgr.24512) 705 : cluster [DBG] pgmap v1216: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:46.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:45 smithi151 bash[22565]: cluster 2024-09-10T18:52:44.799680+0000 mon.a (mon.0) 3536 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:46.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:45 smithi151 bash[22565]: audit 2024-09-10T18:52:44.812912+0000 mon.a (mon.0) 3537 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]': finished 2024-09-10T18:52:46.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:45 smithi151 bash[22565]: cluster 2024-09-10T18:52:44.812968+0000 mon.a (mon.0) 3538 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-09-10T18:52:46.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[18597]: cluster 2024-09-10T18:52:44.238082+0000 mgr.y (mgr.24512) 705 : cluster [DBG] pgmap v1216: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[18597]: cluster 2024-09-10T18:52:44.799680+0000 mon.a (mon.0) 3536 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[18597]: audit 2024-09-10T18:52:44.812912+0000 mon.a (mon.0) 3537 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]': finished 2024-09-10T18:52:46.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[18597]: cluster 2024-09-10T18:52:44.812968+0000 mon.a (mon.0) 3538 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-09-10T18:52:46.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[31607]: cluster 2024-09-10T18:52:44.238082+0000 mgr.y (mgr.24512) 705 : cluster [DBG] pgmap v1216: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:46.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[31607]: cluster 2024-09-10T18:52:44.799680+0000 mon.a (mon.0) 3536 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:46.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[31607]: audit 2024-09-10T18:52:44.812912+0000 mon.a (mon.0) 3537 : audit [INF] from='client.? 172.21.15.100:0/3083627511' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "app": "rados"}]': finished 2024-09-10T18:52:46.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:45 smithi100 bash[31607]: cluster 2024-09-10T18:52:44.812968+0000 mon.a (mon.0) 3538 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-09-10T18:52:48.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:47 smithi151 bash[22565]: cluster 2024-09-10T18:52:46.239936+0000 mgr.y (mgr.24512) 706 : cluster [DBG] pgmap v1218: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-09-10T18:52:48.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:47 smithi100 bash[18597]: cluster 2024-09-10T18:52:46.239936+0000 mgr.y (mgr.24512) 706 : cluster [DBG] pgmap v1218: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-09-10T18:52:48.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:47 smithi100 bash[31607]: cluster 2024-09-10T18:52:46.239936+0000 mgr.y (mgr.24512) 706 : cluster [DBG] pgmap v1218: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-09-10T18:52:50.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:49 smithi151 bash[22565]: cluster 2024-09-10T18:52:48.240740+0000 mgr.y (mgr.24512) 707 : cluster [DBG] pgmap v1219: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T18:52:50.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:49 smithi151 bash[22565]: audit 2024-09-10T18:52:48.860420+0000 mon.c (mon.2) 532 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:50.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:49 smithi151 bash[22565]: audit 2024-09-10T18:52:49.319284+0000 mon.c (mon.2) 533 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:50.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:49 smithi151 bash[22565]: audit 2024-09-10T18:52:49.320660+0000 mon.c (mon.2) 534 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:50.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:49 smithi151 bash[22565]: audit 2024-09-10T18:52:49.330101+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[18597]: cluster 2024-09-10T18:52:48.240740+0000 mgr.y (mgr.24512) 707 : cluster [DBG] pgmap v1219: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[18597]: audit 2024-09-10T18:52:48.860420+0000 mon.c (mon.2) 532 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[18597]: audit 2024-09-10T18:52:49.319284+0000 mon.c (mon.2) 533 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[18597]: audit 2024-09-10T18:52:49.320660+0000 mon.c (mon.2) 534 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[18597]: audit 2024-09-10T18:52:49.330101+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[31607]: cluster 2024-09-10T18:52:48.240740+0000 mgr.y (mgr.24512) 707 : cluster [DBG] pgmap v1219: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[31607]: audit 2024-09-10T18:52:48.860420+0000 mon.c (mon.2) 532 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[31607]: audit 2024-09-10T18:52:49.319284+0000 mon.c (mon.2) 533 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[31607]: audit 2024-09-10T18:52:49.320660+0000 mon.c (mon.2) 534 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:50.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:49 smithi100 bash[31607]: audit 2024-09-10T18:52:49.330101+0000 mon.a (mon.0) 3539 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:51] "GET /metrics HTTP/1.1" 200 223867 "" "Prometheus/2.43.0" 2024-09-10T18:52:52.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:51 smithi151 bash[22565]: cluster 2024-09-10T18:52:50.241557+0000 mgr.y (mgr.24512) 708 : cluster [DBG] pgmap v1220: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 687 B/s rd, 17 KiB/s wr, 2 op/s 2024-09-10T18:52:52.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:51 smithi151 bash[22565]: cluster 2024-09-10T18:52:51.166044+0000 mon.a (mon.0) 3540 : cluster [WRN] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' is full (reached quota's max_objects: 10) 2024-09-10T18:52:52.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:51 smithi151 bash[22565]: cluster 2024-09-10T18:52:51.166638+0000 mon.a (mon.0) 3541 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:52:52.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:51 smithi151 bash[22565]: cluster 2024-09-10T18:52:51.175636+0000 mon.a (mon.0) 3542 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-09-10T18:52:52.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[18597]: cluster 2024-09-10T18:52:50.241557+0000 mgr.y (mgr.24512) 708 : cluster [DBG] pgmap v1220: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 687 B/s rd, 17 KiB/s wr, 2 op/s 2024-09-10T18:52:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[18597]: cluster 2024-09-10T18:52:51.166044+0000 mon.a (mon.0) 3540 : cluster [WRN] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' is full (reached quota's max_objects: 10) 2024-09-10T18:52:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[18597]: cluster 2024-09-10T18:52:51.166638+0000 mon.a (mon.0) 3541 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:52:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[18597]: cluster 2024-09-10T18:52:51.175636+0000 mon.a (mon.0) 3542 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-09-10T18:52:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[31607]: cluster 2024-09-10T18:52:50.241557+0000 mgr.y (mgr.24512) 708 : cluster [DBG] pgmap v1220: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 687 B/s rd, 17 KiB/s wr, 2 op/s 2024-09-10T18:52:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[31607]: cluster 2024-09-10T18:52:51.166044+0000 mon.a (mon.0) 3540 : cluster [WRN] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' is full (reached quota's max_objects: 10) 2024-09-10T18:52:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[31607]: cluster 2024-09-10T18:52:51.166638+0000 mon.a (mon.0) 3541 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:52:52.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:51 smithi100 bash[31607]: cluster 2024-09-10T18:52:51.175636+0000 mon.a (mon.0) 3542 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-09-10T18:52:53.422 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:52:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:52:54.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:53 smithi151 bash[22565]: cluster 2024-09-10T18:52:52.242301+0000 mgr.y (mgr.24512) 709 : cluster [DBG] pgmap v1222: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-10T18:52:54.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:53 smithi151 bash[22565]: audit 2024-09-10T18:52:52.927609+0000 mgr.y (mgr.24512) 710 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:53 smithi100 bash[18597]: cluster 2024-09-10T18:52:52.242301+0000 mgr.y (mgr.24512) 709 : cluster [DBG] pgmap v1222: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-10T18:52:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:53 smithi100 bash[18597]: audit 2024-09-10T18:52:52.927609+0000 mgr.y (mgr.24512) 710 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:53 smithi100 bash[31607]: cluster 2024-09-10T18:52:52.242301+0000 mgr.y (mgr.24512) 709 : cluster [DBG] pgmap v1222: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-10T18:52:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:53 smithi100 bash[31607]: audit 2024-09-10T18:52:52.927609+0000 mgr.y (mgr.24512) 710 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:54.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:52:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:52:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:55 smithi151 bash[22565]: cluster 2024-09-10T18:52:54.243058+0000 mgr.y (mgr.24512) 711 : cluster [DBG] pgmap v1223: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-10T18:52:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:55 smithi151 bash[22565]: audit 2024-09-10T18:52:54.350855+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:55.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:55 smithi151 bash[22565]: audit 2024-09-10T18:52:54.354086+0000 mon.c (mon.2) 535 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:55.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:55 smithi100 bash[18597]: cluster 2024-09-10T18:52:54.243058+0000 mgr.y (mgr.24512) 711 : cluster [DBG] pgmap v1223: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-10T18:52:55.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:55 smithi100 bash[18597]: audit 2024-09-10T18:52:54.350855+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:55 smithi100 bash[18597]: audit 2024-09-10T18:52:54.354086+0000 mon.c (mon.2) 535 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:55 smithi100 bash[31607]: cluster 2024-09-10T18:52:54.243058+0000 mgr.y (mgr.24512) 711 : cluster [DBG] pgmap v1223: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-09-10T18:52:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:55 smithi100 bash[31607]: audit 2024-09-10T18:52:54.350855+0000 mon.a (mon.0) 3543 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:52:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:55 smithi100 bash[31607]: audit 2024-09-10T18:52:54.354086+0000 mon.c (mon.2) 535 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:57.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:57 smithi151 bash[22565]: cluster 2024-09-10T18:52:56.244914+0000 mgr.y (mgr.24512) 712 : cluster [DBG] pgmap v1224: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:57.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:57 smithi100 bash[18597]: cluster 2024-09-10T18:52:56.244914+0000 mgr.y (mgr.24512) 712 : cluster [DBG] pgmap v1224: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:57 smithi100 bash[31607]: cluster 2024-09-10T18:52:56.244914+0000 mgr.y (mgr.24512) 712 : cluster [DBG] pgmap v1224: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:59.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:52:59 smithi151 bash[22565]: cluster 2024-09-10T18:52:58.245684+0000 mgr.y (mgr.24512) 713 : cluster [DBG] pgmap v1225: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:59.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:52:59 smithi100 bash[18597]: cluster 2024-09-10T18:52:58.245684+0000 mgr.y (mgr.24512) 713 : cluster [DBG] pgmap v1225: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:59.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:52:59 smithi100 bash[31607]: cluster 2024-09-10T18:52:58.245684+0000 mgr.y (mgr.24512) 713 : cluster [DBG] pgmap v1225: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:53:01.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:01 smithi151 bash[22565]: cluster 2024-09-10T18:53:00.246437+0000 mgr.y (mgr.24512) 714 : cluster [DBG] pgmap v1226: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:53:01.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:01 smithi100 bash[18597]: cluster 2024-09-10T18:53:00.246437+0000 mgr.y (mgr.24512) 714 : cluster [DBG] pgmap v1226: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:53:01.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:53:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:01] "GET /metrics HTTP/1.1" 200 226363 "" "Prometheus/2.43.0" 2024-09-10T18:53:01.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:01 smithi100 bash[31607]: cluster 2024-09-10T18:53:00.246437+0000 mgr.y (mgr.24512) 714 : cluster [DBG] pgmap v1226: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:53:03.367 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:53:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:53:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:03 smithi151 bash[22565]: cluster 2024-09-10T18:53:02.248320+0000 mgr.y (mgr.24512) 715 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 924 B/s rd, 0 op/s 2024-09-10T18:53:03.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:03 smithi151 bash[22565]: audit 2024-09-10T18:53:02.937784+0000 mgr.y (mgr.24512) 716 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:03.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:03 smithi100 bash[18597]: cluster 2024-09-10T18:53:02.248320+0000 mgr.y (mgr.24512) 715 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 924 B/s rd, 0 op/s 2024-09-10T18:53:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:03 smithi100 bash[18597]: audit 2024-09-10T18:53:02.937784+0000 mgr.y (mgr.24512) 716 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:03 smithi100 bash[31607]: cluster 2024-09-10T18:53:02.248320+0000 mgr.y (mgr.24512) 715 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 924 B/s rd, 0 op/s 2024-09-10T18:53:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:03 smithi100 bash[31607]: audit 2024-09-10T18:53:02.937784+0000 mgr.y (mgr.24512) 716 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:04.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:53:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:05.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:05 smithi151 bash[22565]: cluster 2024-09-10T18:53:04.249219+0000 mgr.y (mgr.24512) 717 : cluster [DBG] pgmap v1228: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:05.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:05 smithi100 bash[18597]: cluster 2024-09-10T18:53:04.249219+0000 mgr.y (mgr.24512) 717 : cluster [DBG] pgmap v1228: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:05 smithi100 bash[31607]: cluster 2024-09-10T18:53:04.249219+0000 mgr.y (mgr.24512) 717 : cluster [DBG] pgmap v1228: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:07.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:07 smithi151 bash[22565]: cluster 2024-09-10T18:53:06.251095+0000 mgr.y (mgr.24512) 718 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:07.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:07 smithi100 bash[18597]: cluster 2024-09-10T18:53:06.251095+0000 mgr.y (mgr.24512) 718 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:07 smithi100 bash[31607]: cluster 2024-09-10T18:53:06.251095+0000 mgr.y (mgr.24512) 718 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:09.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:09 smithi151 bash[22565]: cluster 2024-09-10T18:53:08.251880+0000 mgr.y (mgr.24512) 719 : cluster [DBG] pgmap v1230: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:09.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:09 smithi151 bash[22565]: audit 2024-09-10T18:53:09.344467+0000 mon.c (mon.2) 536 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:09 smithi100 bash[18597]: cluster 2024-09-10T18:53:08.251880+0000 mgr.y (mgr.24512) 719 : cluster [DBG] pgmap v1230: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:09 smithi100 bash[18597]: audit 2024-09-10T18:53:09.344467+0000 mon.c (mon.2) 536 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:09 smithi100 bash[31607]: cluster 2024-09-10T18:53:08.251880+0000 mgr.y (mgr.24512) 719 : cluster [DBG] pgmap v1230: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:09 smithi100 bash[31607]: audit 2024-09-10T18:53:09.344467+0000 mon.c (mon.2) 536 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:11.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:11 smithi151 bash[22565]: cluster 2024-09-10T18:53:10.252718+0000 mgr.y (mgr.24512) 720 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:11 smithi100 bash[18597]: cluster 2024-09-10T18:53:10.252718+0000 mgr.y (mgr.24512) 720 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:53:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:11] "GET /metrics HTTP/1.1" 200 226290 "" "Prometheus/2.43.0" 2024-09-10T18:53:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:11 smithi100 bash[31607]: cluster 2024-09-10T18:53:10.252718+0000 mgr.y (mgr.24512) 720 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:13.389 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:53:12 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:53:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:13 smithi151 bash[22565]: cluster 2024-09-10T18:53:12.254647+0000 mgr.y (mgr.24512) 721 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:13.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:13 smithi151 bash[22565]: audit 2024-09-10T18:53:12.949162+0000 mgr.y (mgr.24512) 722 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:13.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:13 smithi100 bash[18597]: cluster 2024-09-10T18:53:12.254647+0000 mgr.y (mgr.24512) 721 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:13.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:13 smithi100 bash[18597]: audit 2024-09-10T18:53:12.949162+0000 mgr.y (mgr.24512) 722 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:13 smithi100 bash[31607]: cluster 2024-09-10T18:53:12.254647+0000 mgr.y (mgr.24512) 721 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:13 smithi100 bash[31607]: audit 2024-09-10T18:53:12.949162+0000 mgr.y (mgr.24512) 722 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:14.922 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:53:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:15.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=139324 2024-09-10T18:53:15.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 3713687d-818c-43e2-97c0-4c4d90a80f33 max_objects 100 2024-09-10T18:53:15.312 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put onemore /etc/passwd 2024-09-10T18:53:15.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- 172.21.15.100:0/3314422190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35481011a0 msgr2=0x7f3548101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 --2- 172.21.15.100:0/3314422190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35481011a0 0x7f3548101610 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f35380049c0 tx=0x7f3538009e30 comp rx=0 tx=0).stop 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- 172.21.15.100:0/3314422190 shutdown_connections 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 --2- 172.21.15.100:0/3314422190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35481011a0 0x7f3548101610 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 --2- 172.21.15.100:0/3314422190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35480fc560 0x7f3548100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 --2- 172.21.15.100:0/3314422190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fbbc0 0x7f35480fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- 172.21.15.100:0/3314422190 >> 172.21.15.100:0/3314422190 conn(0x7f3548071900 msgr2=0x7f35480f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- 172.21.15.100:0/3314422190 shutdown_connections 2024-09-10T18:53:15.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- 172.21.15.100:0/3314422190 wait complete. 2024-09-10T18:53:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 Processor -- start 2024-09-10T18:53:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- start start 2024-09-10T18:53:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35480fbbc0 0x7f3548118410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fc560 0x7f3548118950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35481011a0 0x7f35481136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3548113cc0 con 0x7f35480fc560 2024-09-10T18:53:15.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3548113e30 con 0x7f35481011a0 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f354dcb9700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3548113ff0 con 0x7f35480fbbc0 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fc560 0x7f3548118950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f35477fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35480fbbc0 0x7f3548118410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fc560 0x7f3548118950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45152/0 (socket says 172.21.15.100:45152) 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 -- 172.21.15.100:0/2645024912 learned_addr learned my addr 172.21.15.100:0/2645024912 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3547fff700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35481011a0 0x7f35481136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 -- 172.21.15.100:0/2645024912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35480fbbc0 msgr2=0x7f3548118410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35480fbbc0 0x7f3548118410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 -- 172.21.15.100:0/2645024912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35481011a0 msgr2=0x7f35481136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35481011a0 0x7f35481136c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3546ffd700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35400048f0 con 0x7f35480fc560 2024-09-10T18:53:15.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.424+0000 7f3547fff700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35481011a0 0x7f35481136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:15.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f3546ffd700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fc560 0x7f3548118950 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f3530002ad0 tx=0x7f35300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:15.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530007cc0 con 0x7f35480fc560 2024-09-10T18:53:15.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f353000f040 con 0x7f35480fc560 2024-09-10T18:53:15.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3538005040 con 0x7f35480fc560 2024-09-10T18:53:15.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353000d860 con 0x7f35480fc560 2024-09-10T18:53:15.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3548120360 con 0x7f35480fc560 2024-09-10T18:53:15.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3548066be0 con 0x7f35480fc560 2024-09-10T18:53:15.432 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.428+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f353000dc90 con 0x7f35480fc560 2024-09-10T18:53:15.432 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.432+0000 7f353ffff700 1 --2- 172.21.15.100:0/2645024912 >> v2:172.21.15.100:6800/2155730476 conn(0x7f353406f010 0x7f35340714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:15.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.432+0000 7f35477fe700 1 --2- 172.21.15.100:0/2645024912 >> v2:172.21.15.100:6800/2155730476 conn(0x7f353406f010 0x7f35340714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:15.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.432+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(752..752 src has 1..752) v4 ==== 7787+0+0 (secure 0 0 0) 0x7f353008e7d0 con 0x7f35480fc560 2024-09-10T18:53:15.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.432+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=753}) v3 -- 0x7f353407a340 con 0x7f35480fc560 2024-09-10T18:53:15.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.432+0000 7f35477fe700 1 --2- 172.21.15.100:0/2645024912 >> v2:172.21.15.100:6800/2155730476 conn(0x7f353406f010 0x7f35340714c0 secure :-1 s=READY pgs=3530 cs=0 l=1 rev1=1 crypto rx=0x7f3540007d60 tx=0x7f354000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:15.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.440+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f353005b060 con 0x7f35480fc560 2024-09-10T18:53:15.609 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:15.608+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f35481141b0 con 0x7f35480fc560 2024-09-10T18:53:15.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:15 smithi151 bash[22565]: cluster 2024-09-10T18:53:14.255415+0000 mgr.y (mgr.24512) 723 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:15.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:15 smithi100 bash[31607]: cluster 2024-09-10T18:53:14.255415+0000 mgr.y (mgr.24512) 723 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:15 smithi100 bash[18597]: cluster 2024-09-10T18:53:14.255415+0000 mgr.y (mgr.24512) 723 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:16.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:16.400+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 7 ==== osd_map(753..753 src has 1..753) v4 ==== 620+0+0 (secure 0 0 0) 0x7f3530007e20 con 0x7f35480fc560 2024-09-10T18:53:16.403 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:16.400+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=754}) v3 -- 0x7f353407af10 con 0x7f35480fc560 2024-09-10T18:53:16.408 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:16.404+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v753) v1 ==== 225+0+0 (secure 0 0 0) 0x7f35481141b0 con 0x7f35480fc560 2024-09-10T18:53:16.517 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:16.516+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f35481127f0 con 0x7f35480fc560 2024-09-10T18:53:16.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:16 smithi151 bash[22565]: audit 2024-09-10T18:53:15.610366+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:53:16.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:16 smithi100 bash[18597]: audit 2024-09-10T18:53:15.610366+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:53:16.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:16 smithi100 bash[31607]: audit 2024-09-10T18:53:15.610366+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:53:17.413 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.412+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 9 ==== osd_map(754..754 src has 1..754) v4 ==== 620+0+0 (secure 0 0 0) 0x7f353005ada0 con 0x7f35480fc560 2024-09-10T18:53:17.414 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.412+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=755}) v3 -- 0x7f353407b3e0 con 0x7f35480fc560 2024-09-10T18:53:17.419 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.416+0000 7f353ffff700 1 -- 172.21.15.100:0/2645024912 <== mon.0 v2:172.21.15.100:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v754) v1 ==== 225+0+0 (secure 0 0 0) 0x7f353005aa00 con 0x7f35480fc560 2024-09-10T18:53:17.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 >> v2:172.21.15.100:6800/2155730476 conn(0x7f353406f010 msgr2=0x7f35340714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:17.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 --2- 172.21.15.100:0/2645024912 >> v2:172.21.15.100:6800/2155730476 conn(0x7f353406f010 0x7f35340714c0 secure :-1 s=READY pgs=3530 cs=0 l=1 rev1=1 crypto rx=0x7f3540007d60 tx=0x7f354000b040 comp rx=0 tx=0).stop 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fc560 msgr2=0x7f3548118950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fc560 0x7f3548118950 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f3530002ad0 tx=0x7f35300046b0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 shutdown_connections 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 --2- 172.21.15.100:0/2645024912 >> v2:172.21.15.100:6800/2155730476 conn(0x7f353406f010 0x7f35340714c0 unknown :-1 s=CLOSED pgs=3530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35481011a0 0x7f35481136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35480fc560 0x7f3548118950 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 --2- 172.21.15.100:0/2645024912 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35480fbbc0 0x7f3548118410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 >> 172.21.15.100:0/2645024912 conn(0x7f3548071900 msgr2=0x7f3548071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 shutdown_connections 2024-09-10T18:53:17.424 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:17.420+0000 7f354dcb9700 1 -- 172.21.15.100:0/2645024912 wait complete. 2024-09-10T18:53:17.425 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 100 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:53:17.433 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 139324 2024-09-10T18:53:17.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:17 smithi151 bash[22565]: cluster 2024-09-10T18:53:16.257257+0000 mgr.y (mgr.24512) 724 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:17.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:17 smithi151 bash[22565]: audit 2024-09-10T18:53:16.408056+0000 mon.a (mon.0) 3545 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:53:17.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:17 smithi151 bash[22565]: cluster 2024-09-10T18:53:16.408292+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-09-10T18:53:17.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:17 smithi151 bash[22565]: audit 2024-09-10T18:53:16.518701+0000 mon.a (mon.0) 3547 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:53:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[18597]: cluster 2024-09-10T18:53:16.257257+0000 mgr.y (mgr.24512) 724 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[18597]: audit 2024-09-10T18:53:16.408056+0000 mon.a (mon.0) 3545 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:53:17.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[18597]: cluster 2024-09-10T18:53:16.408292+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-09-10T18:53:17.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[18597]: audit 2024-09-10T18:53:16.518701+0000 mon.a (mon.0) 3547 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:53:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[31607]: cluster 2024-09-10T18:53:16.257257+0000 mgr.y (mgr.24512) 724 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[31607]: audit 2024-09-10T18:53:16.408056+0000 mon.a (mon.0) 3545 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:53:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[31607]: cluster 2024-09-10T18:53:16.408292+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-09-10T18:53:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:17 smithi100 bash[31607]: audit 2024-09-10T18:53:16.518701+0000 mon.a (mon.0) 3547 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:53:18.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:18 smithi100 bash[18597]: audit 2024-09-10T18:53:17.418648+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:53:18.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:18 smithi100 bash[18597]: cluster 2024-09-10T18:53:17.418725+0000 mon.a (mon.0) 3549 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-09-10T18:53:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:18 smithi100 bash[31607]: audit 2024-09-10T18:53:17.418648+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:53:18.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:18 smithi100 bash[31607]: cluster 2024-09-10T18:53:17.418725+0000 mon.a (mon.0) 3549 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-09-10T18:53:18.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:18 smithi151 bash[22565]: audit 2024-09-10T18:53:17.418648+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? 172.21.15.100:0/2645024912' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:53:18.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:18 smithi151 bash[22565]: cluster 2024-09-10T18:53:17.418725+0000 mon.a (mon.0) 3549 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-09-10T18:53:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:19 smithi100 bash[18597]: cluster 2024-09-10T18:53:18.258042+0000 mgr.y (mgr.24512) 725 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:53:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:19 smithi100 bash[31607]: cluster 2024-09-10T18:53:18.258042+0000 mgr.y (mgr.24512) 725 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:53:19.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:19 smithi151 bash[22565]: cluster 2024-09-10T18:53:18.258042+0000 mgr.y (mgr.24512) 725 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:53:21.192 INFO:tasks.workunit.client.0.smithi100.stderr:+ [ 0 -ne 0 ] 2024-09-10T18:53:21.193 INFO:tasks.workunit.client.0.smithi100.stderr:+ true 2024-09-10T18:53:21.193 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put twomore /etc/passwd 2024-09-10T18:53:21.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 3713687d-818c-43e2-97c0-4c4d90a80f33 max_bytes 100 2024-09-10T18:53:21.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- 172.21.15.100:0/3409154958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f4102da0 msgr2=0x7f62f4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:21.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 --2- 172.21.15.100:0/3409154958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f4102da0 0x7f62f4103170 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f62e80049c0 tx=0x7f62e8009e30 comp rx=0 tx=0).stop 2024-09-10T18:53:21.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- 172.21.15.100:0/3409154958 shutdown_connections 2024-09-10T18:53:21.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 --2- 172.21.15.100:0/3409154958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62f40fd690 0x7f62f4109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:21.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 --2- 172.21.15.100:0/3409154958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f4103740 0x7f62f40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:21.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 --2- 172.21.15.100:0/3409154958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f4102da0 0x7f62f4103170 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:21.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- 172.21.15.100:0/3409154958 >> 172.21.15.100:0/3409154958 conn(0x7f62f4074d50 msgr2=0x7f62f4075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:21.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- 172.21.15.100:0/3409154958 shutdown_connections 2024-09-10T18:53:21.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- 172.21.15.100:0/3409154958 wait complete. 2024-09-10T18:53:21.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 Processor -- start 2024-09-10T18:53:21.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- start start 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f40fd690 0x7f62f4108130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62f4102da0 0x7f62f4108670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f4103740 0x7f62f411f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4108f70 con 0x7f62f4103740 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f411fed0 con 0x7f62f4102da0 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62fae7b700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4120010 con 0x7f62f40fd690 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62f8c17700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f40fd690 0x7f62f4108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62f8c17700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f40fd690 0x7f62f4108130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41408/0 (socket says 172.21.15.100:41408) 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62f8c17700 1 -- 172.21.15.100:0/1924857777 learned_addr learned my addr 172.21.15.100:0/1924857777 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62f9418700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f4103740 0x7f62f411f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.336+0000 7f62f3fff700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62f4102da0 0x7f62f4108670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f8c17700 1 -- 172.21.15.100:0/1924857777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62f4102da0 msgr2=0x7f62f4108670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:21.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f8c17700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62f4102da0 0x7f62f4108670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:21.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f8c17700 1 -- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f4103740 msgr2=0x7f62f411f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:21.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f8c17700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f4103740 0x7f62f411f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:21.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f8c17700 1 -- 172.21.15.100:0/1924857777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40048f0 con 0x7f62f40fd690 2024-09-10T18:53:21.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f3fff700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62f4102da0 0x7f62f4108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:21.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f8c17700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f40fd690 0x7f62f4108130 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f62e800b640 tx=0x7f62e8009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:21.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e800e040 con 0x7f62f40fd690 2024-09-10T18:53:21.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f62e8009540 con 0x7f62f40fd690 2024-09-10T18:53:21.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e8005040 con 0x7f62f40fd690 2024-09-10T18:53:21.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8013b80 con 0x7f62f40fd690 2024-09-10T18:53:21.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f41204e0 con 0x7f62f40fd690 2024-09-10T18:53:21.343 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f4101110 con 0x7f62f40fd690 2024-09-10T18:53:21.344 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.340+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f62e80090d0 con 0x7f62f40fd690 2024-09-10T18:53:21.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.344+0000 7f62f17fa700 1 --2- 172.21.15.100:0/1924857777 >> v2:172.21.15.100:6800/2155730476 conn(0x7f62dc06f010 0x7f62dc0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:21.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.344+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(755..755 src has 1..755) v4 ==== 7787+0+0 (secure 0 0 0) 0x7f62e808fb20 con 0x7f62f40fd690 2024-09-10T18:53:21.345 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.344+0000 7f62f3fff700 1 --2- 172.21.15.100:0/1924857777 >> v2:172.21.15.100:6800/2155730476 conn(0x7f62dc06f010 0x7f62dc0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:21.346 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.344+0000 7f62f3fff700 1 --2- 172.21.15.100:0/1924857777 >> v2:172.21.15.100:6800/2155730476 conn(0x7f62dc06f010 0x7f62dc0714c0 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009fd0 tx=0x7f62e4009ca0 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:21.354 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.352+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f62e805c3b0 con 0x7f62f40fd690 2024-09-10T18:53:21.531 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:21.528+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f62f40fe540 con 0x7f62f40fd690 2024-09-10T18:53:21.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:21] "GET /metrics HTTP/1.1" 200 226290 "" "Prometheus/2.43.0" 2024-09-10T18:53:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[18597]: cluster 2024-09-10T18:53:20.258823+0000 mgr.y (mgr.24512) 726 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:53:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[18597]: cluster 2024-09-10T18:53:21.173925+0000 mon.a (mon.0) 3550 : cluster [INF] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:53:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[18597]: cluster 2024-09-10T18:53:21.174448+0000 mon.a (mon.0) 3551 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:53:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[18597]: cluster 2024-09-10T18:53:21.184240+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-09-10T18:53:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[31607]: cluster 2024-09-10T18:53:20.258823+0000 mgr.y (mgr.24512) 726 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:53:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[31607]: cluster 2024-09-10T18:53:21.173925+0000 mon.a (mon.0) 3550 : cluster [INF] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:53:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[31607]: cluster 2024-09-10T18:53:21.174448+0000 mon.a (mon.0) 3551 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:53:21.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:21 smithi100 bash[31607]: cluster 2024-09-10T18:53:21.184240+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-09-10T18:53:21.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:21 smithi151 bash[22565]: cluster 2024-09-10T18:53:20.258823+0000 mgr.y (mgr.24512) 726 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:53:21.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:21 smithi151 bash[22565]: cluster 2024-09-10T18:53:21.173925+0000 mon.a (mon.0) 3550 : cluster [INF] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:53:21.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:21 smithi151 bash[22565]: cluster 2024-09-10T18:53:21.174448+0000 mon.a (mon.0) 3551 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:53:21.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:21 smithi151 bash[22565]: cluster 2024-09-10T18:53:21.184240+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-09-10T18:53:22.448 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:22.444+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v756) v1 ==== 221+0+0 (secure 0 0 0) 0x7f62e805bd50 con 0x7f62f40fd690 2024-09-10T18:53:22.555 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:22.552+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f62f404c430 con 0x7f62f40fd690 2024-09-10T18:53:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:22 smithi100 bash[18597]: audit 2024-09-10T18:53:21.532486+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.100:0/1924857777' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:22.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:22 smithi100 bash[18597]: audit 2024-09-10T18:53:21.533098+0000 mon.a (mon.0) 3553 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:22 smithi100 bash[31607]: audit 2024-09-10T18:53:21.532486+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.100:0/1924857777' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:22.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:22 smithi100 bash[31607]: audit 2024-09-10T18:53:21.533098+0000 mon.a (mon.0) 3553 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:22.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:22 smithi151 bash[22565]: audit 2024-09-10T18:53:21.532486+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.100:0/1924857777' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:22.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:22 smithi151 bash[22565]: audit 2024-09-10T18:53:21.533098+0000 mon.a (mon.0) 3553 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:23.421 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:53:22 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:53:23.449 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.448+0000 7f62f17fa700 1 -- 172.21.15.100:0/1924857777 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v757) v1 ==== 221+0+0 (secure 0 0 0) 0x7f62e805bf30 con 0x7f62f40fd690 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 >> v2:172.21.15.100:6800/2155730476 conn(0x7f62dc06f010 msgr2=0x7f62dc0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 --2- 172.21.15.100:0/1924857777 >> v2:172.21.15.100:6800/2155730476 conn(0x7f62dc06f010 0x7f62dc0714c0 secure :-1 s=READY pgs=3532 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009fd0 tx=0x7f62e4009ca0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f40fd690 msgr2=0x7f62f4108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f40fd690 0x7f62f4108130 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f62e800b640 tx=0x7f62e8009e30 comp rx=0 tx=0).stop 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 shutdown_connections 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 --2- 172.21.15.100:0/1924857777 >> v2:172.21.15.100:6800/2155730476 conn(0x7f62dc06f010 0x7f62dc0714c0 unknown :-1 s=CLOSED pgs=3532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f4103740 0x7f62f411f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62f4102da0 0x7f62f4108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 --2- 172.21.15.100:0/1924857777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62f40fd690 0x7f62f4108130 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.455 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 >> 172.21.15.100:0/1924857777 conn(0x7f62f4074d50 msgr2=0x7f62f40fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:23.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 shutdown_connections 2024-09-10T18:53:23.456 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:23.452+0000 7f62fae7b700 1 -- 172.21.15.100:0/1924857777 wait complete. 2024-09-10T18:53:23.456 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_bytes = 100 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:53:23.465 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-09-10T18:53:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[18597]: cluster 2024-09-10T18:53:22.259580+0000 mgr.y (mgr.24512) 727 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[18597]: audit 2024-09-10T18:53:22.445221+0000 mon.a (mon.0) 3554 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[18597]: cluster 2024-09-10T18:53:22.445394+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[18597]: audit 2024-09-10T18:53:22.557131+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.100:0/1924857777' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[18597]: audit 2024-09-10T18:53:22.557815+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[18597]: audit 2024-09-10T18:53:22.954465+0000 mgr.y (mgr.24512) 728 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[31607]: cluster 2024-09-10T18:53:22.259580+0000 mgr.y (mgr.24512) 727 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[31607]: audit 2024-09-10T18:53:22.445221+0000 mon.a (mon.0) 3554 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[31607]: cluster 2024-09-10T18:53:22.445394+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-09-10T18:53:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[31607]: audit 2024-09-10T18:53:22.557131+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.100:0/1924857777' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:23.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[31607]: audit 2024-09-10T18:53:22.557815+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:23.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:23 smithi100 bash[31607]: audit 2024-09-10T18:53:22.954465+0000 mgr.y (mgr.24512) 728 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:23 smithi151 bash[22565]: cluster 2024-09-10T18:53:22.259580+0000 mgr.y (mgr.24512) 727 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:23 smithi151 bash[22565]: audit 2024-09-10T18:53:22.445221+0000 mon.a (mon.0) 3554 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:53:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:23 smithi151 bash[22565]: cluster 2024-09-10T18:53:22.445394+0000 mon.a (mon.0) 3555 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-09-10T18:53:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:23 smithi151 bash[22565]: audit 2024-09-10T18:53:22.557131+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.100:0/1924857777' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:23 smithi151 bash[22565]: audit 2024-09-10T18:53:22.557815+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:53:23.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:23 smithi151 bash[22565]: audit 2024-09-10T18:53:22.954465+0000 mgr.y (mgr.24512) 728 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[31607]: audit 2024-09-10T18:53:23.448627+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[31607]: cluster 2024-09-10T18:53:23.448815+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[31607]: audit 2024-09-10T18:53:24.352011+0000 mon.a (mon.0) 3559 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[31607]: audit 2024-09-10T18:53:24.356574+0000 mon.c (mon.2) 539 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[18597]: audit 2024-09-10T18:53:23.448627+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[18597]: cluster 2024-09-10T18:53:23.448815+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[18597]: audit 2024-09-10T18:53:24.352011+0000 mon.a (mon.0) 3559 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:24.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:24 smithi100 bash[18597]: audit 2024-09-10T18:53:24.356574+0000 mon.c (mon.2) 539 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:24.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:53:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:24.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:24 smithi151 bash[22565]: audit 2024-09-10T18:53:23.448627+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:53:24.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:24 smithi151 bash[22565]: cluster 2024-09-10T18:53:23.448815+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-09-10T18:53:24.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:24 smithi151 bash[22565]: audit 2024-09-10T18:53:24.352011+0000 mon.a (mon.0) 3559 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:24.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:24 smithi151 bash[22565]: audit 2024-09-10T18:53:24.356574+0000 mon.c (mon.2) 539 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:25.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:25 smithi100 bash[18597]: cluster 2024-09-10T18:53:24.260367+0000 mgr.y (mgr.24512) 729 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:25 smithi100 bash[31607]: cluster 2024-09-10T18:53:24.260367+0000 mgr.y (mgr.24512) 729 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:25.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:25 smithi151 bash[22565]: cluster 2024-09-10T18:53:24.260367+0000 mgr.y (mgr.24512) 729 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 709 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:26.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:26 smithi100 bash[31607]: cluster 2024-09-10T18:53:26.176239+0000 mon.a (mon.0) 3560 : cluster [WRN] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' is full (reached quota's max_bytes: 100 B) 2024-09-10T18:53:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:26 smithi100 bash[31607]: cluster 2024-09-10T18:53:26.176738+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:53:26.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:26 smithi100 bash[31607]: cluster 2024-09-10T18:53:26.186097+0000 mon.a (mon.0) 3562 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-09-10T18:53:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:26 smithi100 bash[18597]: cluster 2024-09-10T18:53:26.176239+0000 mon.a (mon.0) 3560 : cluster [WRN] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' is full (reached quota's max_bytes: 100 B) 2024-09-10T18:53:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:26 smithi100 bash[18597]: cluster 2024-09-10T18:53:26.176738+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:53:26.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:26 smithi100 bash[18597]: cluster 2024-09-10T18:53:26.186097+0000 mon.a (mon.0) 3562 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-09-10T18:53:26.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:26 smithi151 bash[22565]: cluster 2024-09-10T18:53:26.176239+0000 mon.a (mon.0) 3560 : cluster [WRN] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' is full (reached quota's max_bytes: 100 B) 2024-09-10T18:53:26.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:26 smithi151 bash[22565]: cluster 2024-09-10T18:53:26.176738+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:53:26.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:26 smithi151 bash[22565]: cluster 2024-09-10T18:53:26.186097+0000 mon.a (mon.0) 3562 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-09-10T18:53:27.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:27 smithi100 bash[18597]: cluster 2024-09-10T18:53:26.261146+0000 mgr.y (mgr.24512) 730 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1007 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-10T18:53:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:27 smithi100 bash[31607]: cluster 2024-09-10T18:53:26.261146+0000 mgr.y (mgr.24512) 730 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1007 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-10T18:53:27.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:27 smithi151 bash[22565]: cluster 2024-09-10T18:53:26.261146+0000 mgr.y (mgr.24512) 730 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1007 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-10T18:53:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:29 smithi100 bash[18597]: cluster 2024-09-10T18:53:28.261943+0000 mgr.y (mgr.24512) 731 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:53:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:29 smithi100 bash[31607]: cluster 2024-09-10T18:53:28.261943+0000 mgr.y (mgr.24512) 731 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:53:29.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:29 smithi151 bash[22565]: cluster 2024-09-10T18:53:28.261943+0000 mgr.y (mgr.24512) 731 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:53:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:31 smithi100 bash[18597]: cluster 2024-09-10T18:53:30.262696+0000 mgr.y (mgr.24512) 732 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 654 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-09-10T18:53:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:31 smithi100 bash[31607]: cluster 2024-09-10T18:53:30.262696+0000 mgr.y (mgr.24512) 732 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 654 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-09-10T18:53:31.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:53:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:31] "GET /metrics HTTP/1.1" 200 226285 "" "Prometheus/2.43.0" 2024-09-10T18:53:31.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:31 smithi151 bash[22565]: cluster 2024-09-10T18:53:30.262696+0000 mgr.y (mgr.24512) 732 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 654 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-09-10T18:53:33.421 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:53:32 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:53:33.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:33 smithi100 bash[18597]: cluster 2024-09-10T18:53:32.264537+0000 mgr.y (mgr.24512) 733 : cluster [DBG] pgmap v1248: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-10T18:53:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:33 smithi100 bash[18597]: audit 2024-09-10T18:53:32.965696+0000 mgr.y (mgr.24512) 734 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:33 smithi100 bash[31607]: cluster 2024-09-10T18:53:32.264537+0000 mgr.y (mgr.24512) 733 : cluster [DBG] pgmap v1248: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-10T18:53:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:33 smithi100 bash[31607]: audit 2024-09-10T18:53:32.965696+0000 mgr.y (mgr.24512) 734 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:33.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:33 smithi151 bash[22565]: cluster 2024-09-10T18:53:32.264537+0000 mgr.y (mgr.24512) 733 : cluster [DBG] pgmap v1248: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-10T18:53:33.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:33 smithi151 bash[22565]: audit 2024-09-10T18:53:32.965696+0000 mgr.y (mgr.24512) 734 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:34.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:53:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:35 smithi100 bash[18597]: cluster 2024-09-10T18:53:34.265328+0000 mgr.y (mgr.24512) 735 : cluster [DBG] pgmap v1249: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T18:53:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:35 smithi100 bash[31607]: cluster 2024-09-10T18:53:34.265328+0000 mgr.y (mgr.24512) 735 : cluster [DBG] pgmap v1249: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T18:53:35.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:35 smithi151 bash[22565]: cluster 2024-09-10T18:53:34.265328+0000 mgr.y (mgr.24512) 735 : cluster [DBG] pgmap v1249: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T18:53:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:37 smithi100 bash[18597]: cluster 2024-09-10T18:53:36.267129+0000 mgr.y (mgr.24512) 736 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2024-09-10T18:53:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:37 smithi100 bash[31607]: cluster 2024-09-10T18:53:36.267129+0000 mgr.y (mgr.24512) 736 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2024-09-10T18:53:37.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:37 smithi151 bash[22565]: cluster 2024-09-10T18:53:36.267129+0000 mgr.y (mgr.24512) 736 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1015 B/s rd, 0 op/s 2024-09-10T18:53:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:39 smithi100 bash[18597]: cluster 2024-09-10T18:53:38.267870+0000 mgr.y (mgr.24512) 737 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:39 smithi100 bash[18597]: audit 2024-09-10T18:53:39.352385+0000 mon.a (mon.0) 3563 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:39 smithi100 bash[18597]: audit 2024-09-10T18:53:39.354466+0000 mon.c (mon.2) 540 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:39 smithi100 bash[31607]: cluster 2024-09-10T18:53:38.267870+0000 mgr.y (mgr.24512) 737 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:39 smithi100 bash[31607]: audit 2024-09-10T18:53:39.352385+0000 mon.a (mon.0) 3563 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:39 smithi100 bash[31607]: audit 2024-09-10T18:53:39.354466+0000 mon.c (mon.2) 540 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:39.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:39 smithi151 bash[22565]: cluster 2024-09-10T18:53:38.267870+0000 mgr.y (mgr.24512) 737 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:39.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:39 smithi151 bash[22565]: audit 2024-09-10T18:53:39.352385+0000 mon.a (mon.0) 3563 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:39.922 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:39 smithi151 bash[22565]: audit 2024-09-10T18:53:39.354466+0000 mon.c (mon.2) 540 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:41.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:41 smithi100 bash[18597]: cluster 2024-09-10T18:53:40.268612+0000 mgr.y (mgr.24512) 738 : cluster [DBG] pgmap v1252: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:41.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:53:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:41] "GET /metrics HTTP/1.1" 200 226300 "" "Prometheus/2.43.0" 2024-09-10T18:53:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:41 smithi100 bash[31607]: cluster 2024-09-10T18:53:40.268612+0000 mgr.y (mgr.24512) 738 : cluster [DBG] pgmap v1252: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:41 smithi151 bash[22565]: cluster 2024-09-10T18:53:40.268612+0000 mgr.y (mgr.24512) 738 : cluster [DBG] pgmap v1252: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:43.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:53:42 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:53:43.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:43 smithi100 bash[18597]: cluster 2024-09-10T18:53:42.270519+0000 mgr.y (mgr.24512) 739 : cluster [DBG] pgmap v1253: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:43 smithi100 bash[18597]: audit 2024-09-10T18:53:42.976982+0000 mgr.y (mgr.24512) 740 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:43 smithi100 bash[31607]: cluster 2024-09-10T18:53:42.270519+0000 mgr.y (mgr.24512) 739 : cluster [DBG] pgmap v1253: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:43 smithi100 bash[31607]: audit 2024-09-10T18:53:42.976982+0000 mgr.y (mgr.24512) 740 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:43.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:43 smithi151 bash[22565]: cluster 2024-09-10T18:53:42.270519+0000 mgr.y (mgr.24512) 739 : cluster [DBG] pgmap v1253: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:43.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:43 smithi151 bash[22565]: audit 2024-09-10T18:53:42.976982+0000 mgr.y (mgr.24512) 740 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:44.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:53:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:45.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:45 smithi100 bash[18597]: cluster 2024-09-10T18:53:44.271302+0000 mgr.y (mgr.24512) 741 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:45 smithi100 bash[31607]: cluster 2024-09-10T18:53:44.271302+0000 mgr.y (mgr.24512) 741 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:45.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:45 smithi151 bash[22565]: cluster 2024-09-10T18:53:44.271302+0000 mgr.y (mgr.24512) 741 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:47 smithi100 bash[31607]: cluster 2024-09-10T18:53:46.273168+0000 mgr.y (mgr.24512) 742 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:47.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:47 smithi100 bash[18597]: cluster 2024-09-10T18:53:46.273168+0000 mgr.y (mgr.24512) 742 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:47.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:47 smithi151 bash[22565]: cluster 2024-09-10T18:53:46.273168+0000 mgr.y (mgr.24512) 742 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:49 smithi100 bash[31607]: cluster 2024-09-10T18:53:48.273951+0000 mgr.y (mgr.24512) 743 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:49 smithi100 bash[31607]: audit 2024-09-10T18:53:49.384300+0000 mon.c (mon.2) 541 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:49 smithi100 bash[18597]: cluster 2024-09-10T18:53:48.273951+0000 mgr.y (mgr.24512) 743 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:49 smithi100 bash[18597]: audit 2024-09-10T18:53:49.384300+0000 mon.c (mon.2) 541 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:49.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:49 smithi151 bash[22565]: cluster 2024-09-10T18:53:48.273951+0000 mgr.y (mgr.24512) 743 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:49.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:49 smithi151 bash[22565]: audit 2024-09-10T18:53:49.384300+0000 mon.c (mon.2) 541 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:50 smithi100 bash[18597]: audit 2024-09-10T18:53:49.835303+0000 mon.c (mon.2) 542 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:50 smithi100 bash[18597]: audit 2024-09-10T18:53:49.836866+0000 mon.c (mon.2) 543 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:50 smithi100 bash[18597]: audit 2024-09-10T18:53:49.848250+0000 mon.a (mon.0) 3564 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:50 smithi100 bash[31607]: audit 2024-09-10T18:53:49.835303+0000 mon.c (mon.2) 542 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:50 smithi100 bash[31607]: audit 2024-09-10T18:53:49.836866+0000 mon.c (mon.2) 543 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:50 smithi100 bash[31607]: audit 2024-09-10T18:53:49.848250+0000 mon.a (mon.0) 3564 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:50.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:50 smithi151 bash[22565]: audit 2024-09-10T18:53:49.835303+0000 mon.c (mon.2) 542 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:50.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:50 smithi151 bash[22565]: audit 2024-09-10T18:53:49.836866+0000 mon.c (mon.2) 543 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:50.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:50 smithi151 bash[22565]: audit 2024-09-10T18:53:49.848250+0000 mon.a (mon.0) 3564 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:53:51.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:53:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:51] "GET /metrics HTTP/1.1" 200 226300 "" "Prometheus/2.43.0" 2024-09-10T18:53:52.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:51 smithi151 bash[22565]: cluster 2024-09-10T18:53:50.274716+0000 mgr.y (mgr.24512) 744 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:52.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:51 smithi100 bash[31607]: cluster 2024-09-10T18:53:50.274716+0000 mgr.y (mgr.24512) 744 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:52.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:51 smithi100 bash[18597]: cluster 2024-09-10T18:53:50.274716+0000 mgr.y (mgr.24512) 744 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:53:53.421 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:53:52 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:53:53.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=139426 2024-09-10T18:53:53.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 3713687d-818c-43e2-97c0-4c4d90a80f33 max_bytes 0 2024-09-10T18:53:53.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put two /etc/passwd 2024-09-10T18:53:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- 172.21.15.100:0/3242122088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec88102e40 msgr2=0x7fec88103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 --2- 172.21.15.100:0/3242122088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec88102e40 0x7fec88103210 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d60 tx=0x7fec80004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- 172.21.15.100:0/3242122088 shutdown_connections 2024-09-10T18:53:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 --2- 172.21.15.100:0/3242122088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec880657e0 0x7fec88109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 --2- 172.21.15.100:0/3242122088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec881037e0 0x7fec88065100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 --2- 172.21.15.100:0/3242122088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec88102e40 0x7fec88103210 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:53.580 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- 172.21.15.100:0/3242122088 >> 172.21.15.100:0/3242122088 conn(0x7fec88072b40 msgr2=0x7fec88072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- 172.21.15.100:0/3242122088 shutdown_connections 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- 172.21.15.100:0/3242122088 wait complete. 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 Processor -- start 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- start start 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec880657e0 0x7fec881154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec88102e40 0x7fec88115a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec881037e0 0x7fec8810f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8810fcf0 con 0x7fec881037e0 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8810fe60 con 0x7fec88102e40 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8e676700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec88110020 con 0x7fec880657e0 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec87fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec880657e0 0x7fec881154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec87fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec880657e0 0x7fec881154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:38906/0 (socket says 172.21.15.100:38906) 2024-09-10T18:53:53.581 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec87fff700 1 -- 172.21.15.100:0/1357334025 learned_addr learned my addr 172.21.15.100:0/1357334025 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:53:53.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec8cc13700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec881037e0 0x7fec8810f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:53.582 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.575+0000 7fec877fe700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec88102e40 0x7fec88115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:53.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec87fff700 1 -- 172.21.15.100:0/1357334025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec88102e40 msgr2=0x7fec88115a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:53.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec87fff700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec88102e40 0x7fec88115a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:53.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec87fff700 1 -- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec881037e0 msgr2=0x7fec8810f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:53.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec87fff700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec881037e0 0x7fec8810f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:53.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec87fff700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec78005040 con 0x7fec880657e0 2024-09-10T18:53:53.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec8cc13700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec881037e0 0x7fec8810f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:53.583 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec87fff700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec880657e0 0x7fec881154f0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fec8000b740 tx=0x7fec80004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:53.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8000e070 con 0x7fec880657e0 2024-09-10T18:53:53.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fec800097d0 con 0x7fec880657e0 2024-09-10T18:53:53.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec80009940 con 0x7fec880657e0 2024-09-10T18:53:53.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80012d20 con 0x7fec880657e0 2024-09-10T18:53:53.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8811c310 con 0x7fec880657e0 2024-09-10T18:53:53.584 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec881101b0 con 0x7fec880657e0 2024-09-10T18:53:53.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.579+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fec80012410 con 0x7fec880657e0 2024-09-10T18:53:53.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.583+0000 7fec84ff9700 1 --2- 172.21.15.100:0/1357334025 >> v2:172.21.15.100:6800/2155730476 conn(0x7fec7406f010 0x7fec740714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:53.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.583+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(758..758 src has 1..758) v4 ==== 7787+0+0 (secure 0 0 0) 0x7fec80090260 con 0x7fec880657e0 2024-09-10T18:53:53.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.583+0000 7fec877fe700 1 --2- 172.21.15.100:0/1357334025 >> v2:172.21.15.100:6800/2155730476 conn(0x7fec7406f010 0x7fec740714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:53.585 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.583+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=759}) v3 -- 0x7fec7407a2c0 con 0x7fec880657e0 2024-09-10T18:53:53.586 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.583+0000 7fec877fe700 1 --2- 172.21.15.100:0/1357334025 >> v2:172.21.15.100:6800/2155730476 conn(0x7fec7406f010 0x7fec740714c0 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7fec88110eb0 tx=0x7fec7000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:53.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.591+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fec8005c9d0 con 0x7fec880657e0 2024-09-10T18:53:53.767 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.763+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fec8810c310 con 0x7fec880657e0 2024-09-10T18:53:53.854 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.851+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(759..759 src has 1..759) v4 ==== 620+0+0 (secure 0 0 0) 0x7fec800126c0 con 0x7fec880657e0 2024-09-10T18:53:53.855 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.851+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=760}) v3 -- 0x7fec7407ae90 con 0x7fec880657e0 2024-09-10T18:53:53.862 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.859+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v759) v1 ==== 217+0+0 (secure 0 0 0) 0x7fec80017070 con 0x7fec880657e0 2024-09-10T18:53:53.968 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:53.963+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fec8810c310 con 0x7fec880657e0 2024-09-10T18:53:54.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:53 smithi151 bash[22565]: cluster 2024-09-10T18:53:52.276678+0000 mgr.y (mgr.24512) 745 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:54.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:53 smithi151 bash[22565]: audit 2024-09-10T18:53:52.988364+0000 mgr.y (mgr.24512) 746 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:54.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:53 smithi151 bash[22565]: audit 2024-09-10T18:53:53.768018+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.100:0/1357334025' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:54.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:53 smithi151 bash[22565]: audit 2024-09-10T18:53:53.768475+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:54.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[18597]: cluster 2024-09-10T18:53:52.276678+0000 mgr.y (mgr.24512) 745 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[18597]: audit 2024-09-10T18:53:52.988364+0000 mgr.y (mgr.24512) 746 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[18597]: audit 2024-09-10T18:53:53.768018+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.100:0/1357334025' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:54.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[18597]: audit 2024-09-10T18:53:53.768475+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[31607]: cluster 2024-09-10T18:53:52.276678+0000 mgr.y (mgr.24512) 745 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[31607]: audit 2024-09-10T18:53:52.988364+0000 mgr.y (mgr.24512) 746 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[31607]: audit 2024-09-10T18:53:53.768018+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.100:0/1357334025' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:54.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:53 smithi100 bash[31607]: audit 2024-09-10T18:53:53.768475+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:54.869 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:53:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:53:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:54.884 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.879+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 9 ==== osd_map(760..760 src has 1..760) v4 ==== 620+0+0 (secure 0 0 0) 0x7fec8005c710 con 0x7fec880657e0 2024-09-10T18:53:54.884 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.879+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=761}) v3 -- 0x7fec7407b360 con 0x7fec880657e0 2024-09-10T18:53:54.892 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.887+0000 7fec84ff9700 1 -- 172.21.15.100:0/1357334025 <== mon.2 v2:172.21.15.100:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v760) v1 ==== 217+0+0 (secure 0 0 0) 0x7fec8005c370 con 0x7fec880657e0 2024-09-10T18:53:54.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 >> v2:172.21.15.100:6800/2155730476 conn(0x7fec7406f010 msgr2=0x7fec740714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:54.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 --2- 172.21.15.100:0/1357334025 >> v2:172.21.15.100:6800/2155730476 conn(0x7fec7406f010 0x7fec740714c0 secure :-1 s=READY pgs=3534 cs=0 l=1 rev1=1 crypto rx=0x7fec88110eb0 tx=0x7fec7000b040 comp rx=0 tx=0).stop 2024-09-10T18:53:54.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec880657e0 msgr2=0x7fec881154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:54.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec880657e0 0x7fec881154f0 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fec8000b740 tx=0x7fec80004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:54.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 shutdown_connections 2024-09-10T18:53:54.895 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 --2- 172.21.15.100:0/1357334025 >> v2:172.21.15.100:6800/2155730476 conn(0x7fec7406f010 0x7fec740714c0 unknown :-1 s=CLOSED pgs=3534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec881037e0 0x7fec8810f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec88102e40 0x7fec88115a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 --2- 172.21.15.100:0/1357334025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec880657e0 0x7fec881154f0 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 >> 172.21.15.100:0/1357334025 conn(0x7fec88072b40 msgr2=0x7fec88107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:54.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 shutdown_connections 2024-09-10T18:53:54.896 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:54.891+0000 7fec8e676700 1 -- 172.21.15.100:0/1357334025 wait complete. 2024-09-10T18:53:54.896 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_bytes = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:53:54.904 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 3713687d-818c-43e2-97c0-4c4d90a80f33 max_objects 0 2024-09-10T18:53:55.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- 172.21.15.100:0/2572872881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c103740 msgr2=0x7f873c0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:55.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 --2- 172.21.15.100:0/2572872881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c103740 0x7f873c0fcf60 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f87300049c0 tx=0x7f8730009e30 comp rx=0 tx=0).stop 2024-09-10T18:53:55.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- 172.21.15.100:0/2572872881 shutdown_connections 2024-09-10T18:53:55.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 --2- 172.21.15.100:0/2572872881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 0x7f873c109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:55.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 --2- 172.21.15.100:0/2572872881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c103740 0x7f873c0fcf60 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:55.010 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 --2- 172.21.15.100:0/2572872881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f873c102da0 0x7f873c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:55.011 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- 172.21.15.100:0/2572872881 >> 172.21.15.100:0/2572872881 conn(0x7f873c074d50 msgr2=0x7f873c075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:55.011 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- 172.21.15.100:0/2572872881 shutdown_connections 2024-09-10T18:53:55.011 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- 172.21.15.100:0/2572872881 wait complete. 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 Processor -- start 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- start start 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 0x7f873c113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c102da0 0x7f873c1137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f873c103740 0x7f873c10d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c10da60 con 0x7f873c0fd690 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c10dbd0 con 0x7f873c103740 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8742064700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c10dd90 con 0x7f873c102da0 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 0x7f873c113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 0x7f873c113280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49534/0 (socket says 172.21.15.100:49534) 2024-09-10T18:53:55.013 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 -- 172.21.15.100:0/1988552518 learned_addr learned my addr 172.21.15.100:0/1988552518 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873affd700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c102da0 0x7f873c1137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 -- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c102da0 msgr2=0x7f873c1137c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c102da0 0x7f873c1137c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 -- 172.21.15.100:0/1988552518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f873c103740 msgr2=0x7f873c10d410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f873c103740 0x7f873c10d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734005040 con 0x7f873c0fd690 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873affd700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c102da0 0x7f873c1137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f873b7fe700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 0x7f873c113280 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f872c001e30 tx=0x7f872c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.007+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00eb70 con 0x7f873c0fd690 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f872c007d90 con 0x7f873c0fd690 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8730005040 con 0x7f873c0fd690 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0096e0 con 0x7f873c0fd690 2024-09-10T18:53:55.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c11a0d0 con 0x7f873c0fd690 2024-09-10T18:53:55.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c0fe540 con 0x7f873c0fd690 2024-09-10T18:53:55.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f872c007920 con 0x7f873c0fd690 2024-09-10T18:53:55.016 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f8723fff700 1 --2- 172.21.15.100:0/1988552518 >> v2:172.21.15.100:6800/2155730476 conn(0x7f872406f010 0x7f87240714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:55.016 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.011+0000 7f873affd700 1 --2- 172.21.15.100:0/1988552518 >> v2:172.21.15.100:6800/2155730476 conn(0x7f872406f010 0x7f87240714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:55.016 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.015+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(760..760 src has 1..760) v4 ==== 7787+0+0 (secure 0 0 0) 0x7f872c08e7e0 con 0x7f873c0fd690 2024-09-10T18:53:55.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.015+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=761}) v3 -- 0x7f872407a2c0 con 0x7f873c0fd690 2024-09-10T18:53:55.017 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.015+0000 7f873affd700 1 --2- 172.21.15.100:0/1988552518 >> v2:172.21.15.100:6800/2155730476 conn(0x7f872406f010 0x7f87240714c0 secure :-1 s=READY pgs=3535 cs=0 l=1 rev1=1 crypto rx=0x7f873c076390 tx=0x7f8730009e30 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:55.025 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.023+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f872c04e2a0 con 0x7f873c0fd690 2024-09-10T18:53:55.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:54 smithi151 bash[22565]: audit 2024-09-10T18:53:53.862249+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:53:55.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:54 smithi151 bash[22565]: cluster 2024-09-10T18:53:53.862331+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-09-10T18:53:55.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:54 smithi151 bash[22565]: audit 2024-09-10T18:53:53.969664+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.100:0/1357334025' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:55.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:54 smithi151 bash[22565]: audit 2024-09-10T18:53:53.970388+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:55.172 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:54 smithi151 bash[22565]: audit 2024-09-10T18:53:54.345664+0000 mon.c (mon.2) 546 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:55.202 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.199+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f873c04c430 con 0x7f873c0fd690 2024-09-10T18:53:55.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[18597]: audit 2024-09-10T18:53:53.862249+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[18597]: cluster 2024-09-10T18:53:53.862331+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[18597]: audit 2024-09-10T18:53:53.969664+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.100:0/1357334025' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[18597]: audit 2024-09-10T18:53:53.970388+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[18597]: audit 2024-09-10T18:53:54.345664+0000 mon.c (mon.2) 546 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[31607]: audit 2024-09-10T18:53:53.862249+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[31607]: cluster 2024-09-10T18:53:53.862331+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[31607]: audit 2024-09-10T18:53:53.969664+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.100:0/1357334025' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[31607]: audit 2024-09-10T18:53:53.970388+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:53:55.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:54 smithi100 bash[31607]: audit 2024-09-10T18:53:54.345664+0000 mon.c (mon.2) 546 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:55.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.891+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 7 ==== osd_map(761..761 src has 1..761) v4 ==== 620+0+0 (secure 0 0 0) 0x7f872c007bd0 con 0x7f873c0fd690 2024-09-10T18:53:55.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.895+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=762}) v3 -- 0x7f872407ae90 con 0x7f873c0fd690 2024-09-10T18:53:55.905 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:55.903+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v761) v1 ==== 221+0+0 (secure 0 0 0) 0x7f873c04c430 con 0x7f873c0fd690 2024-09-10T18:53:56.014 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.011+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f873c10c290 con 0x7f873c0fd690 2024-09-10T18:53:56.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:55 smithi151 bash[22565]: cluster 2024-09-10T18:53:54.277519+0000 mgr.y (mgr.24512) 747 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:53:56.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:55 smithi151 bash[22565]: audit 2024-09-10T18:53:54.887912+0000 mon.a (mon.0) 3569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:53:56.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:55 smithi151 bash[22565]: cluster 2024-09-10T18:53:54.888091+0000 mon.a (mon.0) 3570 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-09-10T18:53:56.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:55 smithi151 bash[22565]: audit 2024-09-10T18:53:55.203866+0000 mon.a (mon.0) 3571 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:53:56.187 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.183+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 9 ==== osd_map(762..762 src has 1..762) v4 ==== 620+0+0 (secure 0 0 0) 0x7f872c00ecd0 con 0x7f873c0fd690 2024-09-10T18:53:56.187 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.183+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=763}) v3 -- 0x7f872407b4f0 con 0x7f873c0fd690 2024-09-10T18:53:56.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.191+0000 7f8723fff700 1 -- 172.21.15.100:0/1988552518 <== mon.0 v2:172.21.15.100:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v762) v1 ==== 221+0+0 (secure 0 0 0) 0x7f873c04c430 con 0x7f873c0fd690 2024-09-10T18:53:56.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 >> v2:172.21.15.100:6800/2155730476 conn(0x7f872406f010 msgr2=0x7f87240714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:56.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 --2- 172.21.15.100:0/1988552518 >> v2:172.21.15.100:6800/2155730476 conn(0x7f872406f010 0x7f87240714c0 secure :-1 s=READY pgs=3535 cs=0 l=1 rev1=1 crypto rx=0x7f873c076390 tx=0x7f8730009e30 comp rx=0 tx=0).stop 2024-09-10T18:53:56.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 msgr2=0x7f873c113280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:56.200 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 0x7f873c113280 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f872c001e30 tx=0x7f872c009ee0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 shutdown_connections 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 --2- 172.21.15.100:0/1988552518 >> v2:172.21.15.100:6800/2155730476 conn(0x7f872406f010 0x7f87240714c0 unknown :-1 s=CLOSED pgs=3535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f873c103740 0x7f873c10d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c102da0 0x7f873c1137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 --2- 172.21.15.100:0/1988552518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c0fd690 0x7f873c113280 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 >> 172.21.15.100:0/1988552518 conn(0x7f873c074d50 msgr2=0x7f873c1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 shutdown_connections 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.195+0000 7f8742064700 1 -- 172.21.15.100:0/1988552518 wait complete. 2024-09-10T18:53:56.201 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:53:56.210 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 139426 2024-09-10T18:53:56.210 INFO:tasks.workunit.client.0.smithi100.stderr:+ [ 0 -ne 0 ] 2024-09-10T18:53:56.210 INFO:tasks.workunit.client.0.smithi100.stderr:+ true 2024-09-10T18:53:56.210 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put three /etc/passwd 2024-09-10T18:53:56.251 INFO:tasks.workunit.client.0.smithi100.stderr:+ uuidgen 2024-09-10T18:53:56.253 INFO:tasks.workunit.client.0.smithi100.stderr:+ pp=a36ef4a4-357c-465b-9b7d-4dcd9b201084 2024-09-10T18:53:56.253 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool create a36ef4a4-357c-465b-9b7d-4dcd9b201084 12 2024-09-10T18:53:56.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[31607]: cluster 2024-09-10T18:53:54.277519+0000 mgr.y (mgr.24512) 747 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:53:56.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[31607]: audit 2024-09-10T18:53:54.887912+0000 mon.a (mon.0) 3569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:53:56.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[31607]: cluster 2024-09-10T18:53:54.888091+0000 mon.a (mon.0) 3570 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-09-10T18:53:56.293 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[31607]: audit 2024-09-10T18:53:55.203866+0000 mon.a (mon.0) 3571 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:53:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[18597]: cluster 2024-09-10T18:53:54.277519+0000 mgr.y (mgr.24512) 747 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:53:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[18597]: audit 2024-09-10T18:53:54.887912+0000 mon.a (mon.0) 3569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:53:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[18597]: cluster 2024-09-10T18:53:54.888091+0000 mon.a (mon.0) 3570 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-09-10T18:53:56.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:55 smithi100 bash[18597]: audit 2024-09-10T18:53:55.203866+0000 mon.a (mon.0) 3571 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:53:56.358 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- 172.21.15.100:0/1458817064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb698070480 msgr2=0x7fb6980708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 --2- 172.21.15.100:0/1458817064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb698070480 0x7fb6980708f0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fb688009d60 tx=0x7fb688004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- 172.21.15.100:0/1458817064 shutdown_connections 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 --2- 172.21.15.100:0/1458817064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6981094e0 0x7fb69810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 --2- 172.21.15.100:0/1458817064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb698070480 0x7fb6980708f0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 --2- 172.21.15.100:0/1458817064 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb698071ce0 0x7fb69806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- 172.21.15.100:0/1458817064 >> 172.21.15.100:0/1458817064 conn(0x7fb698073e30 msgr2=0x7fb698074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- 172.21.15.100:0/1458817064 shutdown_connections 2024-09-10T18:53:56.359 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- 172.21.15.100:0/1458817064 wait complete. 2024-09-10T18:53:56.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 Processor -- start 2024-09-10T18:53:56.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- start start 2024-09-10T18:53:56.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698070480 0x7fb6981153d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:56.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6981094e0 0x7fb698115910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:56.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69810f5a0 0x7fb69810fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:56.361 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69810ff80 con 0x7fb69810f5a0 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6981100f0 con 0x7fb698070480 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69e42b700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6981102b0 con 0x7fb6981094e0 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69c9c8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69810f5a0 0x7fb69810fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb6977fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6981094e0 0x7fb698115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69c9c8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69810f5a0 0x7fb69810fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49594/0 (socket says 172.21.15.100:49594) 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb69c9c8700 1 -- 172.21.15.100:0/287119584 learned_addr learned my addr 172.21.15.100:0/287119584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb697fff700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698070480 0x7fb6981153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb6977fe700 1 -- 172.21.15.100:0/287119584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698070480 msgr2=0x7fb6981153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb6977fe700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698070480 0x7fb6981153d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.362 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb6977fe700 1 -- 172.21.15.100:0/287119584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69810f5a0 msgr2=0x7fb69810fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb6977fe700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69810f5a0 0x7fb69810fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb6977fe700 1 -- 172.21.15.100:0/287119584 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688009940 con 0x7fb6981094e0 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.355+0000 7fb697fff700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698070480 0x7fb6981153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb6977fe700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6981094e0 0x7fb698115910 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fb688009110 tx=0x7fb688004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800e070 con 0x7fb6981094e0 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69811c260 con 0x7fb6981094e0 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fb6880097d0 con 0x7fb6981094e0 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68801cd30 con 0x7fb6981094e0 2024-09-10T18:53:56.363 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69811c6c0 con 0x7fb6981094e0 2024-09-10T18:53:56.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb698104260 con 0x7fb6981094e0 2024-09-10T18:53:56.364 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.359+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fb688007ca0 con 0x7fb6981094e0 2024-09-10T18:53:56.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.363+0000 7fb694ff9700 1 --2- 172.21.15.100:0/287119584 >> v2:172.21.15.100:6800/2155730476 conn(0x7fb68006f010 0x7fb6800714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:56.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.363+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(762..762 src has 1..762) v4 ==== 7787+0+0 (secure 0 0 0) 0x7fb688090600 con 0x7fb6981094e0 2024-09-10T18:53:56.365 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.363+0000 7fb697fff700 1 --2- 172.21.15.100:0/287119584 >> v2:172.21.15.100:6800/2155730476 conn(0x7fb68006f010 0x7fb6800714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:56.366 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.363+0000 7fb697fff700 1 --2- 172.21.15.100:0/287119584 >> v2:172.21.15.100:6800/2155730476 conn(0x7fb68006f010 0x7fb6800714c0 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7fb68c004600 tx=0x7fb68c00b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:56.374 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.371+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb68805cd70 con 0x7fb6981094e0 2024-09-10T18:53:56.554 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:56.551+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12} v 0) v1 -- 0x7fb698110a80 con 0x7fb6981094e0 2024-09-10T18:53:57.201 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.199+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]=0 pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' created v763) v1 ==== 176+0+0 (secure 0 0 0) 0x7fb688004810 con 0x7fb6981094e0 2024-09-10T18:53:57.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[18597]: audit 2024-09-10T18:53:55.902886+0000 mon.a (mon.0) 3572 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:53:57.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[18597]: cluster 2024-09-10T18:53:55.903024+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[18597]: audit 2024-09-10T18:53:56.015280+0000 mon.a (mon.0) 3574 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[18597]: cluster 2024-09-10T18:53:56.184165+0000 mon.a (mon.0) 3575 : cluster [INF] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[18597]: cluster 2024-09-10T18:53:56.184614+0000 mon.a (mon.0) 3576 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[18597]: audit 2024-09-10T18:53:56.194806+0000 mon.a (mon.0) 3577 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[18597]: cluster 2024-09-10T18:53:56.194978+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[31607]: audit 2024-09-10T18:53:55.902886+0000 mon.a (mon.0) 3572 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[31607]: cluster 2024-09-10T18:53:55.903024+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[31607]: audit 2024-09-10T18:53:56.015280+0000 mon.a (mon.0) 3574 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[31607]: cluster 2024-09-10T18:53:56.184165+0000 mon.a (mon.0) 3575 : cluster [INF] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[31607]: cluster 2024-09-10T18:53:56.184614+0000 mon.a (mon.0) 3576 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[31607]: audit 2024-09-10T18:53:56.194806+0000 mon.a (mon.0) 3577 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:53:57.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:56 smithi100 bash[31607]: cluster 2024-09-10T18:53:56.194978+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-09-10T18:53:57.307 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.303+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12} v 0) v1 -- 0x7fb69810e430 con 0x7fb6981094e0 2024-09-10T18:53:57.308 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb694ff9700 1 -- 172.21.15.100:0/287119584 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]=0 pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' already exists v763) v1 ==== 183+0+0 (secure 0 0 0) 0x7fb688017070 con 0x7fb6981094e0 2024-09-10T18:53:57.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 >> v2:172.21.15.100:6800/2155730476 conn(0x7fb68006f010 msgr2=0x7fb6800714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 --2- 172.21.15.100:0/287119584 >> v2:172.21.15.100:6800/2155730476 conn(0x7fb68006f010 0x7fb6800714c0 secure :-1 s=READY pgs=3537 cs=0 l=1 rev1=1 crypto rx=0x7fb68c004600 tx=0x7fb68c00b040 comp rx=0 tx=0).stop 2024-09-10T18:53:57.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6981094e0 msgr2=0x7fb698115910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6981094e0 0x7fb698115910 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7fb688009110 tx=0x7fb688004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:57.311 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 shutdown_connections 2024-09-10T18:53:57.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 --2- 172.21.15.100:0/287119584 >> v2:172.21.15.100:6800/2155730476 conn(0x7fb68006f010 0x7fb6800714c0 unknown :-1 s=CLOSED pgs=3537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb69810f5a0 0x7fb69810fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6981094e0 0x7fb698115910 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 --2- 172.21.15.100:0/287119584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb698070480 0x7fb6981153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 >> 172.21.15.100:0/287119584 conn(0x7fb698073e30 msgr2=0x7fb6981013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:57.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 shutdown_connections 2024-09-10T18:53:57.312 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.307+0000 7fb69e42b700 1 -- 172.21.15.100:0/287119584 wait complete. 2024-09-10T18:53:57.313 INFO:tasks.workunit.client.0.smithi100.stderr:pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' already exists 2024-09-10T18:53:57.321 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool application enable a36ef4a4-357c-465b-9b7d-4dcd9b201084 rados 2024-09-10T18:53:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:56 smithi151 bash[22565]: audit 2024-09-10T18:53:55.902886+0000 mon.a (mon.0) 3572 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:53:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:56 smithi151 bash[22565]: cluster 2024-09-10T18:53:55.903024+0000 mon.a (mon.0) 3573 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-09-10T18:53:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:56 smithi151 bash[22565]: audit 2024-09-10T18:53:56.015280+0000 mon.a (mon.0) 3574 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:53:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:56 smithi151 bash[22565]: cluster 2024-09-10T18:53:56.184165+0000 mon.a (mon.0) 3575 : cluster [INF] pool '3713687d-818c-43e2-97c0-4c4d90a80f33' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:53:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:56 smithi151 bash[22565]: cluster 2024-09-10T18:53:56.184614+0000 mon.a (mon.0) 3576 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:53:57.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:56 smithi151 bash[22565]: audit 2024-09-10T18:53:56.194806+0000 mon.a (mon.0) 3577 : audit [INF] from='client.? 172.21.15.100:0/1988552518' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:53:57.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:56 smithi151 bash[22565]: cluster 2024-09-10T18:53:56.194978+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-09-10T18:53:57.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- 172.21.15.100:0/2895463626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d00109700 msgr2=0x7f3d0010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.425 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 --2- 172.21.15.100:0/2895463626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d00109700 0x7f3d0010bae0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009d60 tx=0x7f3cf8004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- 172.21.15.100:0/2895463626 shutdown_connections 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 --2- 172.21.15.100:0/2895463626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d00109700 0x7f3d0010bae0 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 --2- 172.21.15.100:0/2895463626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d000fc460 0x7f3d001091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 --2- 172.21.15.100:0/2895463626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d000fbb50 0x7f3d000fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- 172.21.15.100:0/2895463626 >> 172.21.15.100:0/2895463626 conn(0x7f3d00071930 msgr2=0x7f3d000f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- 172.21.15.100:0/2895463626 shutdown_connections 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- 172.21.15.100:0/2895463626 wait complete. 2024-09-10T18:53:57.426 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 Processor -- start 2024-09-10T18:53:57.427 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- start start 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d000fbb50 0x7f3d00115210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d000fc460 0x7f3d00115750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d00109700 0x7f3d0010f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0010f9f0 con 0x7f3d00109700 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0010fb60 con 0x7f3d000fc460 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d0748c700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0010fd20 con 0x7f3d000fbb50 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05228700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d000fbb50 0x7f3d00115210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05228700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d000fbb50 0x7f3d00115210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:38984/0 (socket says 172.21.15.100:38984) 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05228700 1 -- 172.21.15.100:0/3112250728 learned_addr learned my addr 172.21.15.100:0/3112250728 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05a29700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d00109700 0x7f3d0010f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d04a27700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d000fc460 0x7f3d00115750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:57.429 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05a29700 1 -- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d000fbb50 msgr2=0x7f3d00115210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05a29700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d000fbb50 0x7f3d00115210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05a29700 1 -- 172.21.15.100:0/3112250728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d000fc460 msgr2=0x7f3d00115750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05a29700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d000fc460 0x7f3d00115750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05a29700 1 -- 172.21.15.100:0/3112250728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc005040 con 0x7f3d00109700 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05228700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d000fbb50 0x7f3d00115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.423+0000 7f3d05a29700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d00109700 0x7f3d0010f3a0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009110 tx=0x7f3cf8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.427+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800e030 con 0x7f3d00109700 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.427+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f3cf80097d0 con 0x7f3d00109700 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.427+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf8009940 con 0x7f3d00109700 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.427+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8012410 con 0x7f3d00109700 2024-09-10T18:53:57.430 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.427+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0011c060 con 0x7f3d00109700 2024-09-10T18:53:57.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.427+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d001038d0 con 0x7f3d00109700 2024-09-10T18:53:57.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.427+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f3cf801c650 con 0x7f3d00109700 2024-09-10T18:53:57.432 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.431+0000 7f3cf5ffb700 1 --2- 172.21.15.100:0/3112250728 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3cec06f010 0x7f3cec0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.431+0000 7f3d05228700 1 --2- 172.21.15.100:0/3112250728 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3cec06f010 0x7f3cec0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:57.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.431+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(763..763 src has 1..763) v4 ==== 8162+0+0 (secure 0 0 0) 0x7f3cf8090cf0 con 0x7f3d00109700 2024-09-10T18:53:57.433 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.431+0000 7f3d05228700 1 --2- 172.21.15.100:0/3112250728 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3cec06f010 0x7f3cec0714c0 secure :-1 s=READY pgs=3538 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00041d0 tx=0x7f3cf000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:57.440 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.435+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3cf8095070 con 0x7f3d00109700 2024-09-10T18:53:57.602 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:57.599+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"} v 0) v1 -- 0x7f3d0004c430 con 0x7f3d00109700 2024-09-10T18:53:58.208 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:58.203+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]=0 enabled application 'rados' on pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' v764) v1 ==== 213+0+0 (secure 0 0 0) 0x7f3cf80170d0 con 0x7f3d00109700 2024-09-10T18:53:58.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: cluster 2024-09-10T18:53:56.278255+0000 mgr.y (mgr.24512) 748 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:58.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: audit 2024-09-10T18:53:56.555187+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.100:0/287119584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: audit 2024-09-10T18:53:56.555871+0000 mon.a (mon.0) 3579 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: audit 2024-09-10T18:53:57.201202+0000 mon.a (mon.0) 3580 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]': finished 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: cluster 2024-09-10T18:53:57.201461+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: audit 2024-09-10T18:53:57.308454+0000 mon.c (mon.2) 548 : audit [INF] from='client.? 172.21.15.100:0/287119584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: audit 2024-09-10T18:53:57.308974+0000 mon.a (mon.0) 3582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[18597]: audit 2024-09-10T18:53:57.603413+0000 mon.a (mon.0) 3583 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]: dispatch 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: cluster 2024-09-10T18:53:56.278255+0000 mgr.y (mgr.24512) 748 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: audit 2024-09-10T18:53:56.555187+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.100:0/287119584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: audit 2024-09-10T18:53:56.555871+0000 mon.a (mon.0) 3579 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: audit 2024-09-10T18:53:57.201202+0000 mon.a (mon.0) 3580 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]': finished 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: cluster 2024-09-10T18:53:57.201461+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-09-10T18:53:58.294 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: audit 2024-09-10T18:53:57.308454+0000 mon.c (mon.2) 548 : audit [INF] from='client.? 172.21.15.100:0/287119584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: audit 2024-09-10T18:53:57.308974+0000 mon.a (mon.0) 3582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.295 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:57 smithi100 bash[31607]: audit 2024-09-10T18:53:57.603413+0000 mon.a (mon.0) 3583 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]: dispatch 2024-09-10T18:53:58.313 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:58.311+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"} v 0) v1 -- 0x7f3d0010e450 con 0x7f3d00109700 2024-09-10T18:53:58.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: cluster 2024-09-10T18:53:56.278255+0000 mgr.y (mgr.24512) 748 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:58.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: audit 2024-09-10T18:53:56.555187+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.100:0/287119584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: audit 2024-09-10T18:53:56.555871+0000 mon.a (mon.0) 3579 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: audit 2024-09-10T18:53:57.201202+0000 mon.a (mon.0) 3580 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]': finished 2024-09-10T18:53:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: cluster 2024-09-10T18:53:57.201461+0000 mon.a (mon.0) 3581 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-09-10T18:53:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: audit 2024-09-10T18:53:57.308454+0000 mon.c (mon.2) 548 : audit [INF] from='client.? 172.21.15.100:0/287119584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: audit 2024-09-10T18:53:57.308974+0000 mon.a (mon.0) 3582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pg_num": 12}]: dispatch 2024-09-10T18:53:58.422 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:57 smithi151 bash[22565]: audit 2024-09-10T18:53:57.603413+0000 mon.a (mon.0) 3583 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]: dispatch 2024-09-10T18:53:59.217 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.215+0000 7f3cf5ffb700 1 -- 172.21.15.100:0/3112250728 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]=0 enabled application 'rados' on pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' v765) v1 ==== 213+0+0 (secure 0 0 0) 0x7f3cf8020020 con 0x7f3d00109700 2024-09-10T18:53:59.222 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3cec06f010 msgr2=0x7f3cec0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:59.222 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 --2- 172.21.15.100:0/3112250728 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3cec06f010 0x7f3cec0714c0 secure :-1 s=READY pgs=3538 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00041d0 tx=0x7f3cf000b040 comp rx=0 tx=0).stop 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d00109700 msgr2=0x7f3d0010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d00109700 0x7f3d0010f3a0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009110 tx=0x7f3cf8004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 shutdown_connections 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 --2- 172.21.15.100:0/3112250728 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3cec06f010 0x7f3cec0714c0 unknown :-1 s=CLOSED pgs=3538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d00109700 0x7f3d0010f3a0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d000fc460 0x7f3d00115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 --2- 172.21.15.100:0/3112250728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d000fbb50 0x7f3d00115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 >> 172.21.15.100:0/3112250728 conn(0x7f3d00071930 msgr2=0x7f3d00073440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 shutdown_connections 2024-09-10T18:53:59.223 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.219+0000 7f3d0748c700 1 -- 172.21.15.100:0/3112250728 wait complete. 2024-09-10T18:53:59.224 INFO:tasks.workunit.client.0.smithi100.stderr:enabled application 'rados' on pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' 2024-09-10T18:53:59.232 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota a36ef4a4-357c-465b-9b7d-4dcd9b201084 max_objects 10 2024-09-10T18:53:59.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- 172.21.15.100:0/3279162968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c010dc10 msgr2=0x7fc8c010fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 --2- 172.21.15.100:0/3279162968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c010dc10 0x7fc8c010fff0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009d60 tx=0x7fc8b8004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- 172.21.15.100:0/3279162968 shutdown_connections 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 --2- 172.21.15.100:0/3279162968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c010dc10 0x7fc8c010fff0 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 --2- 172.21.15.100:0/3279162968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 0x7fc8c0070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 --2- 172.21.15.100:0/3279162968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8c0071a40 0x7fc8c006feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- 172.21.15.100:0/3279162968 >> 172.21.15.100:0/3279162968 conn(0x7fc8c0073e30 msgr2=0x7fc8c0074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- 172.21.15.100:0/3279162968 shutdown_connections 2024-09-10T18:53:59.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- 172.21.15.100:0/3279162968 wait complete. 2024-09-10T18:53:59.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 Processor -- start 2024-09-10T18:53:59.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- start start 2024-09-10T18:53:59.338 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 0x7fc8c0119600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c0071a40 0x7fc8c0119b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8c010dc10 0x7fc8c01137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0113e80 con 0x7fc8c00703f0 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c0113ff0 con 0x7fc8c010dc10 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.331+0000 7fc8c6720700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01141b0 con 0x7fc8c0071a40 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c0071a40 0x7fc8c0119b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 0x7fc8c0119600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 0x7fc8c0119600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49612/0 (socket says 172.21.15.100:49612) 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bf7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c0071a40 0x7fc8c0119b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:39002/0 (socket says 172.21.15.100:39002) 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 -- 172.21.15.100:0/1452609868 learned_addr learned my addr 172.21.15.100:0/1452609868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8c4cbd700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8c010dc10 0x7fc8c01137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 -- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c0071a40 msgr2=0x7fc8c0119b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:59.339 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c0071a40 0x7fc8c0119b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 -- 172.21.15.100:0/1452609868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8c010dc10 msgr2=0x7fc8c01137b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8c010dc10 0x7fc8c01137b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 -- 172.21.15.100:0/1452609868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b8009940 con 0x7fc8c00703f0 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bf7fe700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c0071a40 0x7fc8c0119b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bffff700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 0x7fc8c0119600 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4002ad0 tx=0x7fc8b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4004e70 con 0x7fc8c00703f0 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7fc8b400f040 con 0x7fc8c00703f0 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c0114490 con 0x7fc8c00703f0 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b400d830 con 0x7fc8c00703f0 2024-09-10T18:53:59.340 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c01205c0 con 0x7fc8c00703f0 2024-09-10T18:53:59.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c0071410 con 0x7fc8c00703f0 2024-09-10T18:53:59.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7fc8b400dc60 con 0x7fc8c00703f0 2024-09-10T18:53:59.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.335+0000 7fc8bcff9700 1 --2- 172.21.15.100:0/1452609868 >> v2:172.21.15.100:6800/2155730476 conn(0x7fc8a806f010 0x7fc8a80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:59.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.339+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(765..765 src has 1..765) v4 ==== 8175+0+0 (secure 0 0 0) 0x7fc8b408e9a0 con 0x7fc8c00703f0 2024-09-10T18:53:59.341 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.339+0000 7fc8bf7fe700 1 --2- 172.21.15.100:0/1452609868 >> v2:172.21.15.100:6800/2155730476 conn(0x7fc8a806f010 0x7fc8a80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:59.342 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.339+0000 7fc8bf7fe700 1 --2- 172.21.15.100:0/1452609868 >> v2:172.21.15.100:6800/2155730476 conn(0x7fc8a806f010 0x7fc8a80714c0 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0114ec0 tx=0x7fc8b000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:59.348 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.347+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc8b405b030 con 0x7fc8c00703f0 2024-09-10T18:53:59.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:53:59.523+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc8c0112800 con 0x7fc8c00703f0 2024-09-10T18:53:59.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[31607]: audit 2024-09-10T18:53:58.208170+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]': finished 2024-09-10T18:53:59.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[31607]: cluster 2024-09-10T18:53:58.208266+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-09-10T18:53:59.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[31607]: cluster 2024-09-10T18:53:58.278903+0000 mgr.y (mgr.24512) 749 : cluster [DBG] pgmap v1267: 188 pgs: 12 unknown, 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:53:59.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[31607]: audit 2024-09-10T18:53:58.314915+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]: dispatch 2024-09-10T18:53:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[18597]: audit 2024-09-10T18:53:58.208170+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]': finished 2024-09-10T18:53:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[18597]: cluster 2024-09-10T18:53:58.208266+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-09-10T18:53:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[18597]: cluster 2024-09-10T18:53:58.278903+0000 mgr.y (mgr.24512) 749 : cluster [DBG] pgmap v1267: 188 pgs: 12 unknown, 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:53:59.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:53:59 smithi100 bash[18597]: audit 2024-09-10T18:53:58.314915+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]: dispatch 2024-09-10T18:53:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:59 smithi151 bash[22565]: audit 2024-09-10T18:53:58.208170+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]': finished 2024-09-10T18:53:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:59 smithi151 bash[22565]: cluster 2024-09-10T18:53:58.208266+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-09-10T18:53:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:59 smithi151 bash[22565]: cluster 2024-09-10T18:53:58.278903+0000 mgr.y (mgr.24512) 749 : cluster [DBG] pgmap v1267: 188 pgs: 12 unknown, 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:53:59.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:53:59 smithi151 bash[22565]: audit 2024-09-10T18:53:58.314915+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]: dispatch 2024-09-10T18:54:00.219 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:00.214+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool a36ef4a4-357c-465b-9b7d-4dcd9b201084 v766) v1 ==== 223+0+0 (secure 0 0 0) 0x7fc8b405a9d0 con 0x7fc8c00703f0 2024-09-10T18:54:00.327 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:00.322+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc8c0114680 con 0x7fc8c00703f0 2024-09-10T18:54:00.543 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:00 smithi100 bash[31607]: audit 2024-09-10T18:53:59.218121+0000 mon.a (mon.0) 3587 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]': finished 2024-09-10T18:54:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:00 smithi100 bash[31607]: cluster 2024-09-10T18:53:59.218229+0000 mon.a (mon.0) 3588 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-09-10T18:54:00.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:00 smithi100 bash[31607]: audit 2024-09-10T18:53:59.526675+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:00.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:00 smithi100 bash[18597]: audit 2024-09-10T18:53:59.218121+0000 mon.a (mon.0) 3587 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]': finished 2024-09-10T18:54:00.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:00 smithi100 bash[18597]: cluster 2024-09-10T18:53:59.218229+0000 mon.a (mon.0) 3588 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-09-10T18:54:00.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:00 smithi100 bash[18597]: audit 2024-09-10T18:53:59.526675+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:00 smithi151 bash[22565]: audit 2024-09-10T18:53:59.218121+0000 mon.a (mon.0) 3587 : audit [INF] from='client.? 172.21.15.100:0/3112250728' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "app": "rados"}]': finished 2024-09-10T18:54:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:00 smithi151 bash[22565]: cluster 2024-09-10T18:53:59.218229+0000 mon.a (mon.0) 3588 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-09-10T18:54:00.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:00 smithi151 bash[22565]: audit 2024-09-10T18:53:59.526675+0000 mon.a (mon.0) 3589 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:01.240 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.234+0000 7fc8bcff9700 1 -- 172.21.15.100:0/1452609868 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool a36ef4a4-357c-465b-9b7d-4dcd9b201084 v767) v1 ==== 223+0+0 (secure 0 0 0) 0x7fc8b405abb0 con 0x7fc8c00703f0 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 >> v2:172.21.15.100:6800/2155730476 conn(0x7fc8a806f010 msgr2=0x7fc8a80714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 --2- 172.21.15.100:0/1452609868 >> v2:172.21.15.100:6800/2155730476 conn(0x7fc8a806f010 0x7fc8a80714c0 secure :-1 s=READY pgs=3539 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0114ec0 tx=0x7fc8b000b040 comp rx=0 tx=0).stop 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 msgr2=0x7fc8c0119600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 0x7fc8c0119600 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4002ad0 tx=0x7fc8b4004600 comp rx=0 tx=0).stop 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 shutdown_connections 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 --2- 172.21.15.100:0/1452609868 >> v2:172.21.15.100:6800/2155730476 conn(0x7fc8a806f010 0x7fc8a80714c0 unknown :-1 s=CLOSED pgs=3539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8c010dc10 0x7fc8c01137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8c0071a40 0x7fc8c0119b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 --2- 172.21.15.100:0/1452609868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c00703f0 0x7fc8c0119600 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:01.246 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 >> 172.21.15.100:0/1452609868 conn(0x7fc8c0073e30 msgr2=0x7fc8c010e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:01.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 shutdown_connections 2024-09-10T18:54:01.247 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:54:01.242+0000 7fc8c6720700 1 -- 172.21.15.100:0/1452609868 wait complete. 2024-09-10T18:54:01.247 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 10 for pool a36ef4a4-357c-465b-9b7d-4dcd9b201084 2024-09-10T18:54:01.257 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-09-10T18:54:01.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[31607]: audit 2024-09-10T18:54:00.219223+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:01.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[31607]: cluster 2024-09-10T18:54:00.219465+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-09-10T18:54:01.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[31607]: cluster 2024-09-10T18:54:00.279588+0000 mgr.y (mgr.24512) 750 : cluster [DBG] pgmap v1270: 188 pgs: 12 unknown, 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:01.545 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[31607]: audit 2024-09-10T18:54:00.328286+0000 mon.a (mon.0) 3592 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:01.545 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:01] "GET /metrics HTTP/1.1" 200 226270 "" "Prometheus/2.43.0" 2024-09-10T18:54:01.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[18597]: audit 2024-09-10T18:54:00.219223+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:01.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[18597]: cluster 2024-09-10T18:54:00.219465+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-09-10T18:54:01.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[18597]: cluster 2024-09-10T18:54:00.279588+0000 mgr.y (mgr.24512) 750 : cluster [DBG] pgmap v1270: 188 pgs: 12 unknown, 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:01.545 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:01 smithi100 bash[18597]: audit 2024-09-10T18:54:00.328286+0000 mon.a (mon.0) 3592 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:01 smithi151 bash[22565]: audit 2024-09-10T18:54:00.219223+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:01 smithi151 bash[22565]: cluster 2024-09-10T18:54:00.219465+0000 mon.a (mon.0) 3591 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-09-10T18:54:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:01 smithi151 bash[22565]: cluster 2024-09-10T18:54:00.279588+0000 mgr.y (mgr.24512) 750 : cluster [DBG] pgmap v1270: 188 pgs: 12 unknown, 176 active+clean; 735 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:01.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:01 smithi151 bash[22565]: audit 2024-09-10T18:54:00.328286+0000 mon.a (mon.0) 3592 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:02.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:02 smithi100 bash[18597]: audit 2024-09-10T18:54:01.239278+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:02.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:02 smithi100 bash[18597]: cluster 2024-09-10T18:54:01.239324+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-09-10T18:54:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:02 smithi100 bash[31607]: audit 2024-09-10T18:54:01.239278+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:02.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:02 smithi100 bash[31607]: cluster 2024-09-10T18:54:01.239324+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-09-10T18:54:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:02 smithi151 bash[22565]: audit 2024-09-10T18:54:01.239278+0000 mon.a (mon.0) 3593 : audit [INF] from='client.? 172.21.15.100:0/1452609868' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:02 smithi151 bash[22565]: cluster 2024-09-10T18:54:01.239324+0000 mon.a (mon.0) 3594 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-09-10T18:54:03.421 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:54:02 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:54:03.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:03 smithi151 bash[22565]: cluster 2024-09-10T18:54:02.280415+0000 mgr.y (mgr.24512) 751 : cluster [DBG] pgmap v1272: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.4 KiB/s wr, 1 op/s 2024-09-10T18:54:03.421 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:03 smithi151 bash[22565]: audit 2024-09-10T18:54:02.999678+0000 mgr.y (mgr.24512) 752 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:03.543 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:03 smithi100 bash[18597]: cluster 2024-09-10T18:54:02.280415+0000 mgr.y (mgr.24512) 751 : cluster [DBG] pgmap v1272: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.4 KiB/s wr, 1 op/s 2024-09-10T18:54:03.544 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:03 smithi100 bash[18597]: audit 2024-09-10T18:54:02.999678+0000 mgr.y (mgr.24512) 752 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:03 smithi100 bash[31607]: cluster 2024-09-10T18:54:02.280415+0000 mgr.y (mgr.24512) 751 : cluster [DBG] pgmap v1272: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.4 KiB/s wr, 1 op/s 2024-09-10T18:54:03.544 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:03 smithi100 bash[31607]: audit 2024-09-10T18:54:02.999678+0000 mgr.y (mgr.24512) 752 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:04.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:54:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:05 smithi151 bash[22565]: cluster 2024-09-10T18:54:04.281223+0000 mgr.y (mgr.24512) 753 : cluster [DBG] pgmap v1273: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:54:05.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:05 smithi100 bash[18597]: cluster 2024-09-10T18:54:04.281223+0000 mgr.y (mgr.24512) 753 : cluster [DBG] pgmap v1273: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:54:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:05 smithi100 bash[31607]: cluster 2024-09-10T18:54:04.281223+0000 mgr.y (mgr.24512) 753 : cluster [DBG] pgmap v1273: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:54:07.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:07 smithi151 bash[22565]: cluster 2024-09-10T18:54:06.283214+0000 mgr.y (mgr.24512) 754 : cluster [DBG] pgmap v1274: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.7 KiB/s wr, 1 op/s 2024-09-10T18:54:07.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:07 smithi100 bash[18597]: cluster 2024-09-10T18:54:06.283214+0000 mgr.y (mgr.24512) 754 : cluster [DBG] pgmap v1274: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.7 KiB/s wr, 1 op/s 2024-09-10T18:54:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:07 smithi100 bash[31607]: cluster 2024-09-10T18:54:06.283214+0000 mgr.y (mgr.24512) 754 : cluster [DBG] pgmap v1274: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.7 KiB/s wr, 1 op/s 2024-09-10T18:54:09.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:09 smithi151 bash[22565]: cluster 2024-09-10T18:54:08.284075+0000 mgr.y (mgr.24512) 755 : cluster [DBG] pgmap v1275: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:54:09.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:09 smithi100 bash[18597]: cluster 2024-09-10T18:54:08.284075+0000 mgr.y (mgr.24512) 755 : cluster [DBG] pgmap v1275: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:54:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:09 smithi100 bash[31607]: cluster 2024-09-10T18:54:08.284075+0000 mgr.y (mgr.24512) 755 : cluster [DBG] pgmap v1275: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:54:10.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:10 smithi151 bash[22565]: audit 2024-09-10T18:54:09.359452+0000 mon.a (mon.0) 3595 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:10.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:10 smithi151 bash[22565]: audit 2024-09-10T18:54:09.361110+0000 mon.c (mon.2) 549 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:10.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:10 smithi100 bash[18597]: audit 2024-09-10T18:54:09.359452+0000 mon.a (mon.0) 3595 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:10.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:10 smithi100 bash[18597]: audit 2024-09-10T18:54:09.361110+0000 mon.c (mon.2) 549 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:10 smithi100 bash[31607]: audit 2024-09-10T18:54:09.359452+0000 mon.a (mon.0) 3595 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:10.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:10 smithi100 bash[31607]: audit 2024-09-10T18:54:09.361110+0000 mon.c (mon.2) 549 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:11.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:11 smithi151 bash[22565]: cluster 2024-09-10T18:54:10.284867+0000 mgr.y (mgr.24512) 756 : cluster [DBG] pgmap v1276: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T18:54:11.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:11 smithi100 bash[18597]: cluster 2024-09-10T18:54:10.284867+0000 mgr.y (mgr.24512) 756 : cluster [DBG] pgmap v1276: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T18:54:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:11 smithi100 bash[31607]: cluster 2024-09-10T18:54:10.284867+0000 mgr.y (mgr.24512) 756 : cluster [DBG] pgmap v1276: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T18:54:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:54:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:11] "GET /metrics HTTP/1.1" 200 228745 "" "Prometheus/2.43.0" 2024-09-10T18:54:13.375 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:54:13 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:54:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:13 smithi151 bash[22565]: cluster 2024-09-10T18:54:12.286876+0000 mgr.y (mgr.24512) 757 : cluster [DBG] pgmap v1277: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 926 B/s rd, 0 op/s 2024-09-10T18:54:13.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:13 smithi151 bash[22565]: audit 2024-09-10T18:54:13.011113+0000 mgr.y (mgr.24512) 758 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:13 smithi100 bash[18597]: cluster 2024-09-10T18:54:12.286876+0000 mgr.y (mgr.24512) 757 : cluster [DBG] pgmap v1277: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 926 B/s rd, 0 op/s 2024-09-10T18:54:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:13 smithi100 bash[18597]: audit 2024-09-10T18:54:13.011113+0000 mgr.y (mgr.24512) 758 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:13 smithi100 bash[31607]: cluster 2024-09-10T18:54:12.286876+0000 mgr.y (mgr.24512) 757 : cluster [DBG] pgmap v1277: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 926 B/s rd, 0 op/s 2024-09-10T18:54:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:13 smithi100 bash[31607]: audit 2024-09-10T18:54:13.011113+0000 mgr.y (mgr.24512) 758 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:14.921 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:54:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:15.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:15 smithi151 bash[22565]: cluster 2024-09-10T18:54:14.287666+0000 mgr.y (mgr.24512) 759 : cluster [DBG] pgmap v1278: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:15.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:15 smithi100 bash[18597]: cluster 2024-09-10T18:54:14.287666+0000 mgr.y (mgr.24512) 759 : cluster [DBG] pgmap v1278: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:15 smithi100 bash[31607]: cluster 2024-09-10T18:54:14.287666+0000 mgr.y (mgr.24512) 759 : cluster [DBG] pgmap v1278: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:17.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:17 smithi151 bash[22565]: cluster 2024-09-10T18:54:16.289634+0000 mgr.y (mgr.24512) 760 : cluster [DBG] pgmap v1279: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:17.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:17 smithi100 bash[18597]: cluster 2024-09-10T18:54:16.289634+0000 mgr.y (mgr.24512) 760 : cluster [DBG] pgmap v1279: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:17 smithi100 bash[31607]: cluster 2024-09-10T18:54:16.289634+0000 mgr.y (mgr.24512) 760 : cluster [DBG] pgmap v1279: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:19.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:19 smithi151 bash[22565]: cluster 2024-09-10T18:54:18.290475+0000 mgr.y (mgr.24512) 761 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:19.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:19 smithi100 bash[18597]: cluster 2024-09-10T18:54:18.290475+0000 mgr.y (mgr.24512) 761 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:19 smithi100 bash[31607]: cluster 2024-09-10T18:54:18.290475+0000 mgr.y (mgr.24512) 761 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:21.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:21 smithi151 bash[22565]: cluster 2024-09-10T18:54:20.291298+0000 mgr.y (mgr.24512) 762 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:21.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:21 smithi100 bash[18597]: cluster 2024-09-10T18:54:20.291298+0000 mgr.y (mgr.24512) 762 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:21.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:54:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:21] "GET /metrics HTTP/1.1" 200 228745 "" "Prometheus/2.43.0" 2024-09-10T18:54:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:21 smithi100 bash[31607]: cluster 2024-09-10T18:54:20.291298+0000 mgr.y (mgr.24512) 762 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:23.398 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:54:23 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:54:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:23 smithi151 bash[22565]: cluster 2024-09-10T18:54:22.293300+0000 mgr.y (mgr.24512) 763 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:23.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:23 smithi151 bash[22565]: audit 2024-09-10T18:54:23.022471+0000 mgr.y (mgr.24512) 764 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:23 smithi100 bash[18597]: cluster 2024-09-10T18:54:22.293300+0000 mgr.y (mgr.24512) 763 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:23 smithi100 bash[18597]: audit 2024-09-10T18:54:23.022471+0000 mgr.y (mgr.24512) 764 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:23 smithi100 bash[31607]: cluster 2024-09-10T18:54:22.293300+0000 mgr.y (mgr.24512) 763 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:23 smithi100 bash[31607]: audit 2024-09-10T18:54:23.022471+0000 mgr.y (mgr.24512) 764 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:24.670 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:54:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:24.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:24 smithi151 bash[22565]: audit 2024-09-10T18:54:24.346704+0000 mon.c (mon.2) 550 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:24.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:24 smithi100 bash[18597]: audit 2024-09-10T18:54:24.346704+0000 mon.c (mon.2) 550 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:24 smithi100 bash[31607]: audit 2024-09-10T18:54:24.346704+0000 mon.c (mon.2) 550 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:25.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:25 smithi151 bash[22565]: cluster 2024-09-10T18:54:24.294124+0000 mgr.y (mgr.24512) 765 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:25.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:25 smithi100 bash[18597]: cluster 2024-09-10T18:54:24.294124+0000 mgr.y (mgr.24512) 765 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:25 smithi100 bash[31607]: cluster 2024-09-10T18:54:24.294124+0000 mgr.y (mgr.24512) 765 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:27.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:27 smithi151 bash[22565]: cluster 2024-09-10T18:54:26.296167+0000 mgr.y (mgr.24512) 766 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:27.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:27 smithi100 bash[18597]: cluster 2024-09-10T18:54:26.296167+0000 mgr.y (mgr.24512) 766 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:27 smithi100 bash[31607]: cluster 2024-09-10T18:54:26.296167+0000 mgr.y (mgr.24512) 766 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:29.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:29 smithi151 bash[22565]: cluster 2024-09-10T18:54:28.296997+0000 mgr.y (mgr.24512) 767 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:29.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:29 smithi100 bash[18597]: cluster 2024-09-10T18:54:28.296997+0000 mgr.y (mgr.24512) 767 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:29 smithi100 bash[31607]: cluster 2024-09-10T18:54:28.296997+0000 mgr.y (mgr.24512) 767 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:31.258 INFO:tasks.workunit.client.0.smithi100.stderr:+ seq 1 10 2024-09-10T18:54:31.259 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj1 /etc/passwd 2024-09-10T18:54:31.310 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj2 /etc/passwd 2024-09-10T18:54:31.359 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj3 /etc/passwd 2024-09-10T18:54:31.406 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj4 /etc/passwd 2024-09-10T18:54:31.453 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj5 /etc/passwd 2024-09-10T18:54:31.504 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj6 /etc/passwd 2024-09-10T18:54:31.554 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj7 /etc/passwd 2024-09-10T18:54:31.600 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj8 /etc/passwd 2024-09-10T18:54:31.645 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj9 /etc/passwd 2024-09-10T18:54:31.695 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p a36ef4a4-357c-465b-9b7d-4dcd9b201084 put obj10 /etc/passwd 2024-09-10T18:54:31.742 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-09-10T18:54:31.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:54:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:31] "GET /metrics HTTP/1.1" 200 228759 "" "Prometheus/2.43.0" 2024-09-10T18:54:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:31 smithi100 bash[18597]: cluster 2024-09-10T18:54:30.297808+0000 mgr.y (mgr.24512) 768 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:31 smithi100 bash[31607]: cluster 2024-09-10T18:54:30.297808+0000 mgr.y (mgr.24512) 768 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:31.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:31 smithi151 bash[22565]: cluster 2024-09-10T18:54:30.297808+0000 mgr.y (mgr.24512) 768 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:33.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:54:33 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:54:33.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:33 smithi100 bash[18597]: cluster 2024-09-10T18:54:32.299773+0000 mgr.y (mgr.24512) 769 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:33 smithi100 bash[18597]: audit 2024-09-10T18:54:33.028383+0000 mgr.y (mgr.24512) 770 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:33 smithi100 bash[31607]: cluster 2024-09-10T18:54:32.299773+0000 mgr.y (mgr.24512) 769 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:33.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:33 smithi100 bash[31607]: audit 2024-09-10T18:54:33.028383+0000 mgr.y (mgr.24512) 770 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:33.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:33 smithi151 bash[22565]: cluster 2024-09-10T18:54:32.299773+0000 mgr.y (mgr.24512) 769 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-10T18:54:33.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:33 smithi151 bash[22565]: audit 2024-09-10T18:54:33.028383+0000 mgr.y (mgr.24512) 770 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:34.920 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:54:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:35.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:35 smithi100 bash[18597]: cluster 2024-09-10T18:54:34.300587+0000 mgr.y (mgr.24512) 771 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:35 smithi100 bash[31607]: cluster 2024-09-10T18:54:34.300587+0000 mgr.y (mgr.24512) 771 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:35.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:35 smithi151 bash[22565]: cluster 2024-09-10T18:54:34.300587+0000 mgr.y (mgr.24512) 771 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 760 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:37.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:37 smithi100 bash[18597]: cluster 2024-09-10T18:54:36.302597+0000 mgr.y (mgr.24512) 772 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T18:54:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:37 smithi100 bash[31607]: cluster 2024-09-10T18:54:36.302597+0000 mgr.y (mgr.24512) 772 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T18:54:37.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:37 smithi151 bash[22565]: cluster 2024-09-10T18:54:36.302597+0000 mgr.y (mgr.24512) 772 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T18:54:39.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:39 smithi100 bash[18597]: cluster 2024-09-10T18:54:38.303422+0000 mgr.y (mgr.24512) 773 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:54:39.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:39 smithi100 bash[18597]: audit 2024-09-10T18:54:39.347102+0000 mon.c (mon.2) 551 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:39 smithi100 bash[31607]: cluster 2024-09-10T18:54:38.303422+0000 mgr.y (mgr.24512) 773 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:54:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:39 smithi100 bash[31607]: audit 2024-09-10T18:54:39.347102+0000 mon.c (mon.2) 551 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:39.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:39 smithi151 bash[22565]: cluster 2024-09-10T18:54:38.303422+0000 mgr.y (mgr.24512) 773 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:54:39.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:39 smithi151 bash[22565]: audit 2024-09-10T18:54:39.347102+0000 mon.c (mon.2) 551 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:41.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:41] "GET /metrics HTTP/1.1" 200 228767 "" "Prometheus/2.43.0" 2024-09-10T18:54:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[18597]: cluster 2024-09-10T18:54:40.304239+0000 mgr.y (mgr.24512) 774 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:54:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[18597]: cluster 2024-09-10T18:54:41.195413+0000 mon.a (mon.0) 3596 : cluster [WRN] pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' is full (reached quota's max_objects: 10) 2024-09-10T18:54:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[18597]: cluster 2024-09-10T18:54:41.195930+0000 mon.a (mon.0) 3597 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:54:41.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[18597]: cluster 2024-09-10T18:54:41.207793+0000 mon.a (mon.0) 3598 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-09-10T18:54:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[31607]: cluster 2024-09-10T18:54:40.304239+0000 mgr.y (mgr.24512) 774 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:54:41.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[31607]: cluster 2024-09-10T18:54:41.195413+0000 mon.a (mon.0) 3596 : cluster [WRN] pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' is full (reached quota's max_objects: 10) 2024-09-10T18:54:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[31607]: cluster 2024-09-10T18:54:41.195930+0000 mon.a (mon.0) 3597 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:54:41.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:41 smithi100 bash[31607]: cluster 2024-09-10T18:54:41.207793+0000 mon.a (mon.0) 3598 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-09-10T18:54:41.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:41 smithi151 bash[22565]: cluster 2024-09-10T18:54:40.304239+0000 mgr.y (mgr.24512) 774 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:54:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:41 smithi151 bash[22565]: cluster 2024-09-10T18:54:41.195413+0000 mon.a (mon.0) 3596 : cluster [WRN] pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' is full (reached quota's max_objects: 10) 2024-09-10T18:54:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:41 smithi151 bash[22565]: cluster 2024-09-10T18:54:41.195930+0000 mon.a (mon.0) 3597 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:54:41.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:41 smithi151 bash[22565]: cluster 2024-09-10T18:54:41.207793+0000 mon.a (mon.0) 3598 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-09-10T18:54:43.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:54:43 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:54:43.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:43 smithi100 bash[31607]: cluster 2024-09-10T18:54:42.305028+0000 mgr.y (mgr.24512) 775 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:54:43.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:43 smithi100 bash[31607]: audit 2024-09-10T18:54:43.039641+0000 mgr.y (mgr.24512) 776 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:43 smithi100 bash[18597]: cluster 2024-09-10T18:54:42.305028+0000 mgr.y (mgr.24512) 775 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:54:43.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:43 smithi100 bash[18597]: audit 2024-09-10T18:54:43.039641+0000 mgr.y (mgr.24512) 776 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:43.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:43 smithi151 bash[22565]: cluster 2024-09-10T18:54:42.305028+0000 mgr.y (mgr.24512) 775 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:54:43.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:43 smithi151 bash[22565]: audit 2024-09-10T18:54:43.039641+0000 mgr.y (mgr.24512) 776 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:44.920 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:54:44 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:45.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:45 smithi100 bash[18597]: cluster 2024-09-10T18:54:44.305851+0000 mgr.y (mgr.24512) 777 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:54:45.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:45 smithi100 bash[31607]: cluster 2024-09-10T18:54:44.305851+0000 mgr.y (mgr.24512) 777 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:54:45.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:45 smithi151 bash[22565]: cluster 2024-09-10T18:54:44.305851+0000 mgr.y (mgr.24512) 777 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:54:47.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:47 smithi100 bash[18597]: cluster 2024-09-10T18:54:46.307757+0000 mgr.y (mgr.24512) 778 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:47.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:47 smithi100 bash[31607]: cluster 2024-09-10T18:54:46.307757+0000 mgr.y (mgr.24512) 778 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:47.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:47 smithi151 bash[22565]: cluster 2024-09-10T18:54:46.307757+0000 mgr.y (mgr.24512) 778 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:49.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:49 smithi100 bash[31607]: cluster 2024-09-10T18:54:48.308541+0000 mgr.y (mgr.24512) 779 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:49.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:49 smithi100 bash[18597]: cluster 2024-09-10T18:54:48.308541+0000 mgr.y (mgr.24512) 779 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:49.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:49 smithi151 bash[22565]: cluster 2024-09-10T18:54:48.308541+0000 mgr.y (mgr.24512) 779 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:50.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:50 smithi100 bash[18597]: audit 2024-09-10T18:54:49.896439+0000 mon.c (mon.2) 552 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:50 smithi100 bash[18597]: audit 2024-09-10T18:54:50.200068+0000 mon.a (mon.0) 3599 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:50.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:50 smithi100 bash[18597]: audit 2024-09-10T18:54:50.212268+0000 mon.a (mon.0) 3600 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:50 smithi100 bash[31607]: audit 2024-09-10T18:54:49.896439+0000 mon.c (mon.2) 552 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:50 smithi100 bash[31607]: audit 2024-09-10T18:54:50.200068+0000 mon.a (mon.0) 3599 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:50.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:50 smithi100 bash[31607]: audit 2024-09-10T18:54:50.212268+0000 mon.a (mon.0) 3600 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:50.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:50 smithi151 bash[22565]: audit 2024-09-10T18:54:49.896439+0000 mon.c (mon.2) 552 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:50.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:50 smithi151 bash[22565]: audit 2024-09-10T18:54:50.200068+0000 mon.a (mon.0) 3599 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:50.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:50 smithi151 bash[22565]: audit 2024-09-10T18:54:50.212268+0000 mon.a (mon.0) 3600 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:51.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:51] "GET /metrics HTTP/1.1" 200 228767 "" "Prometheus/2.43.0" 2024-09-10T18:54:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[31607]: cluster 2024-09-10T18:54:50.309036+0000 mgr.y (mgr.24512) 780 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[31607]: audit 2024-09-10T18:54:50.679407+0000 mon.c (mon.2) 553 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[31607]: audit 2024-09-10T18:54:50.680974+0000 mon.c (mon.2) 554 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:51.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[31607]: audit 2024-09-10T18:54:50.690755+0000 mon.a (mon.0) 3601 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:51.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[18597]: cluster 2024-09-10T18:54:50.309036+0000 mgr.y (mgr.24512) 780 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:51.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[18597]: audit 2024-09-10T18:54:50.679407+0000 mon.c (mon.2) 553 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:51.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[18597]: audit 2024-09-10T18:54:50.680974+0000 mon.c (mon.2) 554 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:51.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:51 smithi100 bash[18597]: audit 2024-09-10T18:54:50.690755+0000 mon.a (mon.0) 3601 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:51.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:51 smithi151 bash[22565]: cluster 2024-09-10T18:54:50.309036+0000 mgr.y (mgr.24512) 780 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:54:51.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:51 smithi151 bash[22565]: audit 2024-09-10T18:54:50.679407+0000 mon.c (mon.2) 553 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:51.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:51 smithi151 bash[22565]: audit 2024-09-10T18:54:50.680974+0000 mon.c (mon.2) 554 : audit [INF] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:51.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:51 smithi151 bash[22565]: audit 2024-09-10T18:54:50.690755+0000 mon.a (mon.0) 3601 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:53.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:54:53 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:54:53.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:53 smithi100 bash[18597]: cluster 2024-09-10T18:54:52.310932+0000 mgr.y (mgr.24512) 781 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-10T18:54:53.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:53 smithi100 bash[18597]: audit 2024-09-10T18:54:53.042922+0000 mgr.y (mgr.24512) 782 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:53 smithi100 bash[31607]: cluster 2024-09-10T18:54:52.310932+0000 mgr.y (mgr.24512) 781 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-10T18:54:53.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:53 smithi100 bash[31607]: audit 2024-09-10T18:54:53.042922+0000 mgr.y (mgr.24512) 782 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:53.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:53 smithi151 bash[22565]: cluster 2024-09-10T18:54:52.310932+0000 mgr.y (mgr.24512) 781 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-10T18:54:53.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:53 smithi151 bash[22565]: audit 2024-09-10T18:54:53.042922+0000 mgr.y (mgr.24512) 782 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:54.920 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:54:54 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:54:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:55.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:55 smithi151 bash[22565]: cluster 2024-09-10T18:54:54.311686+0000 mgr.y (mgr.24512) 783 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:55 smithi151 bash[22565]: audit 2024-09-10T18:54:54.356281+0000 mon.a (mon.0) 3602 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:55.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:55 smithi151 bash[22565]: audit 2024-09-10T18:54:54.360285+0000 mon.c (mon.2) 555 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:55 smithi100 bash[31607]: cluster 2024-09-10T18:54:54.311686+0000 mgr.y (mgr.24512) 783 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:55 smithi100 bash[31607]: audit 2024-09-10T18:54:54.356281+0000 mon.a (mon.0) 3602 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:55.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:55 smithi100 bash[31607]: audit 2024-09-10T18:54:54.360285+0000 mon.c (mon.2) 555 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:55 smithi100 bash[18597]: cluster 2024-09-10T18:54:54.311686+0000 mgr.y (mgr.24512) 783 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:55 smithi100 bash[18597]: audit 2024-09-10T18:54:54.356281+0000 mon.a (mon.0) 3602 : audit [INF] from='mgr.24512 ' entity='mgr.y' 2024-09-10T18:54:55.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:55 smithi100 bash[18597]: audit 2024-09-10T18:54:54.360285+0000 mon.c (mon.2) 555 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:57.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:57 smithi151 bash[22565]: cluster 2024-09-10T18:54:56.313691+0000 mgr.y (mgr.24512) 784 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:57.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:57 smithi100 bash[31607]: cluster 2024-09-10T18:54:56.313691+0000 mgr.y (mgr.24512) 784 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:57.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:57 smithi100 bash[18597]: cluster 2024-09-10T18:54:56.313691+0000 mgr.y (mgr.24512) 784 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:59.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:54:59 smithi151 bash[22565]: cluster 2024-09-10T18:54:58.314460+0000 mgr.y (mgr.24512) 785 : cluster [DBG] pgmap v1301: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:59.793 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:54:59 smithi100 bash[31607]: cluster 2024-09-10T18:54:58.314460+0000 mgr.y (mgr.24512) 785 : cluster [DBG] pgmap v1301: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:54:59.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:54:59 smithi100 bash[18597]: cluster 2024-09-10T18:54:58.314460+0000 mgr.y (mgr.24512) 785 : cluster [DBG] pgmap v1301: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:01.662 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:01 smithi151 bash[22565]: cluster 2024-09-10T18:55:00.315246+0000 mgr.y (mgr.24512) 786 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:01.743 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 3713687d-818c-43e2-97c0-4c4d90a80f33 put threemore /etc/passwd 2024-09-10T18:55:01.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:01 smithi100 bash[18597]: cluster 2024-09-10T18:55:00.315246+0000 mgr.y (mgr.24512) 786 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:01.746 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:01 smithi100 bash[31607]: cluster 2024-09-10T18:55:00.315246+0000 mgr.y (mgr.24512) 786 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:01.747 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:01 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:01] "GET /metrics HTTP/1.1" 200 228770 "" "Prometheus/2.43.0" 2024-09-10T18:55:01.795 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 3713687d-818c-43e2-97c0-4c4d90a80f33 max_bytes 0 2024-09-10T18:55:01.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- 172.21.15.100:0/223249693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc1037e0 msgr2=0x7f8ebc0fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:01.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/223249693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc1037e0 0x7f8ebc0fcec0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0049c0 tx=0x7f8eac009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:01.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- 172.21.15.100:0/223249693 shutdown_connections 2024-09-10T18:55:01.909 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/223249693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc0fd5f0 0x7f8ebc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/223249693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc1037e0 0x7f8ebc0fcec0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/223249693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc102e40 0x7f8ebc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- 172.21.15.100:0/223249693 >> 172.21.15.100:0/223249693 conn(0x7f8ebc073b80 msgr2=0x7f8ebc0fafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- 172.21.15.100:0/223249693 shutdown_connections 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- 172.21.15.100:0/223249693 wait complete. 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 Processor -- start 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- start start 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc0fd5f0 0x7f8ebc1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc102e40 0x7f8ebc113720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc1037e0 0x7f8ebc10d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:01.910 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc10d9c0 con 0x7f8ebc0fd5f0 2024-09-10T18:55:01.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc10db30 con 0x7f8ebc1037e0 2024-09-10T18:55:01.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ec13bc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc10dcf0 con 0x7f8ebc102e40 2024-09-10T18:55:01.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc102e40 0x7f8ebc113720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:01.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ebaffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc0fd5f0 0x7f8ebc1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:01.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc102e40 0x7f8ebc113720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:53472/0 (socket says 172.21.15.100:53472) 2024-09-10T18:55:01.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 -- 172.21.15.100:0/2977138332 learned_addr learned my addr 172.21.15.100:0/2977138332 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:55:01.911 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ebb7fe700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc1037e0 0x7f8ebc10d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:01.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 -- 172.21.15.100:0/2977138332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc1037e0 msgr2=0x7f8ebc10d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:01.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc1037e0 0x7f8ebc10d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:01.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 -- 172.21.15.100:0/2977138332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc0fd5f0 msgr2=0x7f8ebc1131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:01.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc0fd5f0 0x7f8ebc1131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:01.912 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0005040 con 0x7f8ebc102e40 2024-09-10T18:55:01.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8ebb7fe700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc1037e0 0x7f8ebc10d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:55:01.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.905+0000 7f8eba7fc700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc102e40 0x7f8ebc113720 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f8eac013df0 tx=0x7f8eac009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:01.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00e050 con 0x7f8ebc102e40 2024-09-10T18:55:01.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8eac009750 con 0x7f8ebc102e40 2024-09-10T18:55:01.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac01bbc0 con 0x7f8ebc102e40 2024-09-10T18:55:01.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac005040 con 0x7f8ebc102e40 2024-09-10T18:55:01.913 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc11a030 con 0x7f8ebc102e40 2024-09-10T18:55:01.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc0fe5b0 con 0x7f8ebc102e40 2024-09-10T18:55:01.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f8eac01b3f0 con 0x7f8ebc102e40 2024-09-10T18:55:01.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.909+0000 7f8ea37fe700 1 --2- 172.21.15.100:0/2977138332 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8ea406f010 0x7f8ea40714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:01.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.913+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(768..768 src has 1..768) v4 ==== 8175+0+0 (secure 0 0 0) 0x7f8eac08ff70 con 0x7f8ebc102e40 2024-09-10T18:55:01.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.913+0000 7f8ebaffd700 1 --2- 172.21.15.100:0/2977138332 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8ea406f010 0x7f8ea40714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:01.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.913+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=769}) v3 -- 0x7f8ea407ab60 con 0x7f8ebc102e40 2024-09-10T18:55:01.914 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.913+0000 7f8ebaffd700 1 --2- 172.21.15.100:0/2977138332 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8ea406f010 0x7f8ea40714c0 secure :-1 s=READY pgs=3551 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009fd0 tx=0x7f8eb000b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:01.921 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:01.921+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8eac05c760 con 0x7f8ebc102e40 2024-09-10T18:55:02.096 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:02.093+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f8ebc10e850 con 0x7f8ebc102e40 2024-09-10T18:55:02.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:02.389+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(769..769 src has 1..769) v4 ==== 620+0+0 (secure 0 0 0) 0x7f8eac05c4a0 con 0x7f8ebc102e40 2024-09-10T18:55:02.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:02.389+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=770}) v3 -- 0x7f8ea407b710 con 0x7f8ebc102e40 2024-09-10T18:55:02.391 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:02.389+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v769) v1 ==== 217+0+0 (secure 0 0 0) 0x7f8eac029030 con 0x7f8ebc102e40 2024-09-10T18:55:02.497 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:02.497+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f8ebc10e850 con 0x7f8ebc102e40 2024-09-10T18:55:02.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:02 smithi151 bash[22565]: audit 2024-09-10T18:55:02.097431+0000 mon.c (mon.2) 556 : audit [INF] from='client.? 172.21.15.100:0/2977138332' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:02.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:02 smithi151 bash[22565]: audit 2024-09-10T18:55:02.098040+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:02 smithi100 bash[18597]: audit 2024-09-10T18:55:02.097431+0000 mon.c (mon.2) 556 : audit [INF] from='client.? 172.21.15.100:0/2977138332' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:02.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:02 smithi100 bash[18597]: audit 2024-09-10T18:55:02.098040+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:02.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:02 smithi100 bash[31607]: audit 2024-09-10T18:55:02.097431+0000 mon.c (mon.2) 556 : audit [INF] from='client.? 172.21.15.100:0/2977138332' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:02.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:02 smithi100 bash[31607]: audit 2024-09-10T18:55:02.098040+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:03.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.385+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 9 ==== osd_map(770..770 src has 1..770) v4 ==== 620+0+0 (secure 0 0 0) 0x7f8eac05d090 con 0x7f8ebc102e40 2024-09-10T18:55:03.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.385+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=771}) v3 -- 0x7f8ea407bc20 con 0x7f8ebc102e40 2024-09-10T18:55:03.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.389+0000 7f8ea37fe700 1 -- 172.21.15.100:0/2977138332 <== mon.2 v2:172.21.15.100:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v770) v1 ==== 217+0+0 (secure 0 0 0) 0x7f8eac073060 con 0x7f8ebc102e40 2024-09-10T18:55:03.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8ea406f010 msgr2=0x7f8ea40714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:03.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/2977138332 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8ea406f010 0x7f8ea40714c0 secure :-1 s=READY pgs=3551 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009fd0 tx=0x7f8eb000b040 comp rx=0 tx=0).stop 2024-09-10T18:55:03.396 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc102e40 msgr2=0x7f8ebc113720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc102e40 0x7f8ebc113720 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f8eac013df0 tx=0x7f8eac009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 shutdown_connections 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/2977138332 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8ea406f010 0x7f8ea40714c0 unknown :-1 s=CLOSED pgs=3551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ebc1037e0 0x7f8ebc10d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ebc102e40 0x7f8ebc113720 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 --2- 172.21.15.100:0/2977138332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ebc0fd5f0 0x7f8ebc1131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 >> 172.21.15.100:0/2977138332 conn(0x7f8ebc073b80 msgr2=0x7f8ebc0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:03.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.393+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 shutdown_connections 2024-09-10T18:55:03.398 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.397+0000 7f8ec13bc700 1 -- 172.21.15.100:0/2977138332 wait complete. 2024-09-10T18:55:03.398 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_bytes = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:55:03.399 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:03 smithi151 bash[22565]: cluster 2024-09-10T18:55:02.317286+0000 mgr.y (mgr.24512) 787 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:03.399 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:03 smithi151 bash[22565]: audit 2024-09-10T18:55:02.387538+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:55:03.399 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:03 smithi151 bash[22565]: cluster 2024-09-10T18:55:02.387672+0000 mon.a (mon.0) 3605 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-09-10T18:55:03.399 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:03 smithi151 bash[22565]: audit 2024-09-10T18:55:02.498401+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.100:0/2977138332' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:03.399 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:03 smithi151 bash[22565]: audit 2024-09-10T18:55:02.499076+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:03.399 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:03 smithi151 bash[22565]: audit 2024-09-10T18:55:03.054299+0000 mgr.y (mgr.24512) 788 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:03.399 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:55:03 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:55:03.408 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 3713687d-818c-43e2-97c0-4c4d90a80f33 max_objects 0 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- 172.21.15.100:0/1715698017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c102e00 msgr2=0x7f208c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 --2- 172.21.15.100:0/1715698017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c102e00 0x7f208c1031d0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f20740049c0 tx=0x7f2074009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- 172.21.15.100:0/1715698017 shutdown_connections 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 --2- 172.21.15.100:0/1715698017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f208c0fd700 0x7f208c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 --2- 172.21.15.100:0/1715698017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c103710 0x7f208c0fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 --2- 172.21.15.100:0/1715698017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c102e00 0x7f208c1031d0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- 172.21.15.100:0/1715698017 >> 172.21.15.100:0/1715698017 conn(0x7f208c073e30 msgr2=0x7f208c074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- 172.21.15.100:0/1715698017 shutdown_connections 2024-09-10T18:55:03.506 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- 172.21.15.100:0/1715698017 wait complete. 2024-09-10T18:55:03.507 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 Processor -- start 2024-09-10T18:55:03.507 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- start start 2024-09-10T18:55:03.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c0fd700 0x7f208c113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:03.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f208c102e00 0x7f208c113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:03.508 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c103710 0x7f208c10d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c10da30 con 0x7f208c103710 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c10dba0 con 0x7f208c102e00 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c10dd60 con 0x7f208c0fd700 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c0fd700 0x7f208c113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f209177a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c103710 0x7f208c10d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c0fd700 0x7f208c113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:53486/0 (socket says 172.21.15.100:53486) 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f209177a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c103710 0x7f208c10d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57510/0 (socket says 172.21.15.100:57510) 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 -- 172.21.15.100:0/2366087629 learned_addr learned my addr 172.21.15.100:0/2366087629 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f208bfff700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f208c102e00 0x7f208c113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 -- 172.21.15.100:0/2366087629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f208c102e00 msgr2=0x7f208c113790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f208c102e00 0x7f208c113790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.509 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 -- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c103710 msgr2=0x7f208c10d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c103710 0x7f208c10d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074005040 con 0x7f208c0fd700 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f209177a700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c103710 0x7f208c10d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f2090f79700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c0fd700 0x7f208c113250 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f2074014ea0 tx=0x7f2074009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207400e040 con 0x7f208c0fd700 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f2074009690 con 0x7f208c0fd700 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208c10dfe0 con 0x7f208c0fd700 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207401ca00 con 0x7f208c0fd700 2024-09-10T18:55:03.510 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.505+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c11a240 con 0x7f208c0fd700 2024-09-10T18:55:03.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.509+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c0fe550 con 0x7f208c0fd700 2024-09-10T18:55:03.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.509+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f2074009220 con 0x7f208c0fd700 2024-09-10T18:55:03.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.509+0000 7f20897fa700 1 --2- 172.21.15.100:0/2366087629 >> v2:172.21.15.100:6800/2155730476 conn(0x7f207806f010 0x7f20780714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:03.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.509+0000 7f208bfff700 1 --2- 172.21.15.100:0/2366087629 >> v2:172.21.15.100:6800/2155730476 conn(0x7f207806f010 0x7f20780714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:03.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.509+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(770..770 src has 1..770) v4 ==== 8175+0+0 (secure 0 0 0) 0x7f207408fba0 con 0x7f208c0fd700 2024-09-10T18:55:03.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.509+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=771}) v3 -- 0x7f207807aae0 con 0x7f208c0fd700 2024-09-10T18:55:03.512 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.509+0000 7f208bfff700 1 --2- 172.21.15.100:0/2366087629 >> v2:172.21.15.100:6800/2155730476 conn(0x7f207806f010 0x7f20780714c0 secure :-1 s=READY pgs=3552 cs=0 l=1 rev1=1 crypto rx=0x7f208c10eb10 tx=0x7f208400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:03.520 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.517+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f207405c210 con 0x7f208c0fd700 2024-09-10T18:55:03.692 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:03.689+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f208c10c210 con 0x7f208c0fd700 2024-09-10T18:55:03.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[18597]: cluster 2024-09-10T18:55:02.317286+0000 mgr.y (mgr.24512) 787 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[18597]: audit 2024-09-10T18:55:02.387538+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[18597]: cluster 2024-09-10T18:55:02.387672+0000 mon.a (mon.0) 3605 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[18597]: audit 2024-09-10T18:55:02.498401+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.100:0/2977138332' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[18597]: audit 2024-09-10T18:55:02.499076+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[18597]: audit 2024-09-10T18:55:03.054299+0000 mgr.y (mgr.24512) 788 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[31607]: cluster 2024-09-10T18:55:02.317286+0000 mgr.y (mgr.24512) 787 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[31607]: audit 2024-09-10T18:55:02.387538+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[31607]: cluster 2024-09-10T18:55:02.387672+0000 mon.a (mon.0) 3605 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-09-10T18:55:03.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[31607]: audit 2024-09-10T18:55:02.498401+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.100:0/2977138332' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[31607]: audit 2024-09-10T18:55:02.499076+0000 mon.a (mon.0) 3606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:55:03.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:03 smithi100 bash[31607]: audit 2024-09-10T18:55:03.054299+0000 mgr.y (mgr.24512) 788 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:04.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:04.417+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(771..771 src has 1..771) v4 ==== 620+0+0 (secure 0 0 0) 0x7f20740094d0 con 0x7f208c0fd700 2024-09-10T18:55:04.418 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:04.417+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=772}) v3 -- 0x7f207807b560 con 0x7f208c0fd700 2024-09-10T18:55:04.423 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:04.421+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v771) v1 ==== 221+0+0 (secure 0 0 0) 0x7f207405bbb0 con 0x7f208c0fd700 2024-09-10T18:55:04.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:04.525+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f208c04c430 con 0x7f208c0fd700 2024-09-10T18:55:04.670 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:55:04 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:04 smithi151 bash[22565]: audit 2024-09-10T18:55:03.390595+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:55:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:04 smithi151 bash[22565]: cluster 2024-09-10T18:55:03.390790+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-09-10T18:55:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:04 smithi151 bash[22565]: audit 2024-09-10T18:55:03.693751+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.100:0/2366087629' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:04.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:04 smithi151 bash[22565]: audit 2024-09-10T18:55:03.694443+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[31607]: audit 2024-09-10T18:55:03.390595+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[31607]: cluster 2024-09-10T18:55:03.390790+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[31607]: audit 2024-09-10T18:55:03.693751+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.100:0/2366087629' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[31607]: audit 2024-09-10T18:55:03.694443+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[18597]: audit 2024-09-10T18:55:03.390595+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[18597]: cluster 2024-09-10T18:55:03.390790+0000 mon.a (mon.0) 3608 : cluster [DBG] osdmap e770: 8 total, 8 up, 8 in 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[18597]: audit 2024-09-10T18:55:03.693751+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.100:0/2366087629' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:04.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:04 smithi100 bash[18597]: audit 2024-09-10T18:55:03.694443+0000 mon.a (mon.0) 3609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:05.431 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.429+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 9 ==== osd_map(772..772 src has 1..772) v4 ==== 620+0+0 (secure 0 0 0) 0x7f2074009800 con 0x7f208c0fd700 2024-09-10T18:55:05.432 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.429+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=773}) v3 -- 0x7f207807ba70 con 0x7f208c0fd700 2024-09-10T18:55:05.437 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.437+0000 7f20897fa700 1 -- 172.21.15.100:0/2366087629 <== mon.2 v2:172.21.15.100:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 v772) v1 ==== 221+0+0 (secure 0 0 0) 0x7f2074068060 con 0x7f208c0fd700 2024-09-10T18:55:05.441 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 >> v2:172.21.15.100:6800/2155730476 conn(0x7f207806f010 msgr2=0x7f20780714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 --2- 172.21.15.100:0/2366087629 >> v2:172.21.15.100:6800/2155730476 conn(0x7f207806f010 0x7f20780714c0 secure :-1 s=READY pgs=3552 cs=0 l=1 rev1=1 crypto rx=0x7f208c10eb10 tx=0x7f208400b040 comp rx=0 tx=0).stop 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c0fd700 msgr2=0x7f208c113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c0fd700 0x7f208c113250 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f2074014ea0 tx=0x7f2074009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 shutdown_connections 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 --2- 172.21.15.100:0/2366087629 >> v2:172.21.15.100:6800/2155730476 conn(0x7f207806f010 0x7f20780714c0 unknown :-1 s=CLOSED pgs=3552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f208c103710 0x7f208c10d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f208c102e00 0x7f208c113790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 --2- 172.21.15.100:0/2366087629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f208c0fd700 0x7f208c113250 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 >> 172.21.15.100:0/2366087629 conn(0x7f208c073e30 msgr2=0x7f208c1009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 shutdown_connections 2024-09-10T18:55:05.442 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:05.441+0000 7f20931dd700 1 -- 172.21.15.100:0/2366087629 wait complete. 2024-09-10T18:55:05.443 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 0 for pool 3713687d-818c-43e2-97c0-4c4d90a80f33 2024-09-10T18:55:05.452 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-09-10T18:55:05.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:05 smithi151 bash[22565]: cluster 2024-09-10T18:55:04.318125+0000 mgr.y (mgr.24512) 789 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:55:05.670 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:05 smithi151 bash[22565]: audit 2024-09-10T18:55:04.416239+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:55:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:05 smithi151 bash[22565]: cluster 2024-09-10T18:55:04.416342+0000 mon.a (mon.0) 3611 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-09-10T18:55:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:05 smithi151 bash[22565]: audit 2024-09-10T18:55:04.528198+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.100:0/2366087629' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:05.671 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:05 smithi151 bash[22565]: audit 2024-09-10T18:55:04.528855+0000 mon.a (mon.0) 3612 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[18597]: cluster 2024-09-10T18:55:04.318125+0000 mgr.y (mgr.24512) 789 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[18597]: audit 2024-09-10T18:55:04.416239+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[18597]: cluster 2024-09-10T18:55:04.416342+0000 mon.a (mon.0) 3611 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[18597]: audit 2024-09-10T18:55:04.528198+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.100:0/2366087629' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[18597]: audit 2024-09-10T18:55:04.528855+0000 mon.a (mon.0) 3612 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[31607]: cluster 2024-09-10T18:55:04.318125+0000 mgr.y (mgr.24512) 789 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 887 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[31607]: audit 2024-09-10T18:55:04.416239+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:55:05.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[31607]: cluster 2024-09-10T18:55:04.416342+0000 mon.a (mon.0) 3611 : cluster [DBG] osdmap e771: 8 total, 8 up, 8 in 2024-09-10T18:55:05.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[31607]: audit 2024-09-10T18:55:04.528198+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.100:0/2366087629' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:05.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:05 smithi100 bash[31607]: audit 2024-09-10T18:55:04.528855+0000 mon.a (mon.0) 3612 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:55:06.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:06 smithi100 bash[18597]: audit 2024-09-10T18:55:05.434473+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:55:06.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:06 smithi100 bash[18597]: cluster 2024-09-10T18:55:05.434578+0000 mon.a (mon.0) 3614 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-09-10T18:55:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:06 smithi100 bash[31607]: audit 2024-09-10T18:55:05.434473+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:55:06.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:06 smithi100 bash[31607]: cluster 2024-09-10T18:55:05.434578+0000 mon.a (mon.0) 3614 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-09-10T18:55:06.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:06 smithi151 bash[22565]: audit 2024-09-10T18:55:05.434473+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:55:06.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:06 smithi151 bash[22565]: cluster 2024-09-10T18:55:05.434578+0000 mon.a (mon.0) 3614 : cluster [DBG] osdmap e772: 8 total, 8 up, 8 in 2024-09-10T18:55:07.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:07 smithi100 bash[18597]: cluster 2024-09-10T18:55:06.318920+0000 mgr.y (mgr.24512) 790 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:55:07.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:07 smithi100 bash[31607]: cluster 2024-09-10T18:55:06.318920+0000 mgr.y (mgr.24512) 790 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:55:07.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:07 smithi151 bash[22565]: cluster 2024-09-10T18:55:06.318920+0000 mgr.y (mgr.24512) 790 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:55:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:09 smithi100 bash[18597]: cluster 2024-09-10T18:55:08.319723+0000 mgr.y (mgr.24512) 791 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-10T18:55:09.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:09 smithi100 bash[18597]: audit 2024-09-10T18:55:09.347685+0000 mon.c (mon.2) 560 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:09 smithi100 bash[31607]: cluster 2024-09-10T18:55:08.319723+0000 mgr.y (mgr.24512) 791 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-10T18:55:09.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:09 smithi100 bash[31607]: audit 2024-09-10T18:55:09.347685+0000 mon.c (mon.2) 560 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:09.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:09 smithi151 bash[22565]: cluster 2024-09-10T18:55:08.319723+0000 mgr.y (mgr.24512) 791 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-10T18:55:09.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:09 smithi151 bash[22565]: audit 2024-09-10T18:55:09.347685+0000 mon.c (mon.2) 560 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:11.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:11 smithi100 bash[18597]: cluster 2024-09-10T18:55:10.320534+0000 mgr.y (mgr.24512) 792 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 738 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-09-10T18:55:11.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:11 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:11] "GET /metrics HTTP/1.1" 200 228777 "" "Prometheus/2.43.0" 2024-09-10T18:55:11.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:11 smithi100 bash[31607]: cluster 2024-09-10T18:55:10.320534+0000 mgr.y (mgr.24512) 792 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 738 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-09-10T18:55:11.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:11 smithi151 bash[22565]: cluster 2024-09-10T18:55:10.320534+0000 mgr.y (mgr.24512) 792 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 738 B/s rd, 1.9 KiB/s wr, 0 op/s 2024-09-10T18:55:13.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:55:13 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:55:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:13 smithi100 bash[18597]: cluster 2024-09-10T18:55:12.322514+0000 mgr.y (mgr.24512) 793 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-10T18:55:13.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:13 smithi100 bash[18597]: audit 2024-09-10T18:55:13.065660+0000 mgr.y (mgr.24512) 794 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:13 smithi100 bash[31607]: cluster 2024-09-10T18:55:12.322514+0000 mgr.y (mgr.24512) 793 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-10T18:55:13.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:13 smithi100 bash[31607]: audit 2024-09-10T18:55:13.065660+0000 mgr.y (mgr.24512) 794 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:13.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:13 smithi151 bash[22565]: cluster 2024-09-10T18:55:12.322514+0000 mgr.y (mgr.24512) 793 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-10T18:55:13.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:13 smithi151 bash[22565]: audit 2024-09-10T18:55:13.065660+0000 mgr.y (mgr.24512) 794 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:14.920 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:55:14 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:15.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:15 smithi100 bash[18597]: cluster 2024-09-10T18:55:14.323322+0000 mgr.y (mgr.24512) 795 : cluster [DBG] pgmap v1313: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-10T18:55:15.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:15 smithi100 bash[31607]: cluster 2024-09-10T18:55:14.323322+0000 mgr.y (mgr.24512) 795 : cluster [DBG] pgmap v1313: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-10T18:55:15.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:15 smithi151 bash[22565]: cluster 2024-09-10T18:55:14.323322+0000 mgr.y (mgr.24512) 795 : cluster [DBG] pgmap v1313: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-10T18:55:17.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:17 smithi100 bash[18597]: cluster 2024-09-10T18:55:16.325265+0000 mgr.y (mgr.24512) 796 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-09-10T18:55:17.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:17 smithi100 bash[31607]: cluster 2024-09-10T18:55:16.325265+0000 mgr.y (mgr.24512) 796 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-09-10T18:55:17.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:17 smithi151 bash[22565]: cluster 2024-09-10T18:55:16.325265+0000 mgr.y (mgr.24512) 796 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-09-10T18:55:19.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:19 smithi100 bash[18597]: cluster 2024-09-10T18:55:18.326078+0000 mgr.y (mgr.24512) 797 : cluster [DBG] pgmap v1315: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:19.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:19 smithi100 bash[31607]: cluster 2024-09-10T18:55:18.326078+0000 mgr.y (mgr.24512) 797 : cluster [DBG] pgmap v1315: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:19.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:19 smithi151 bash[22565]: cluster 2024-09-10T18:55:18.326078+0000 mgr.y (mgr.24512) 797 : cluster [DBG] pgmap v1315: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:21.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:21 smithi100 bash[18597]: cluster 2024-09-10T18:55:20.326842+0000 mgr.y (mgr.24512) 798 : cluster [DBG] pgmap v1316: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:21.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:21 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:21] "GET /metrics HTTP/1.1" 200 228777 "" "Prometheus/2.43.0" 2024-09-10T18:55:21.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:21 smithi100 bash[31607]: cluster 2024-09-10T18:55:20.326842+0000 mgr.y (mgr.24512) 798 : cluster [DBG] pgmap v1316: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:21.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:21 smithi151 bash[22565]: cluster 2024-09-10T18:55:20.326842+0000 mgr.y (mgr.24512) 798 : cluster [DBG] pgmap v1316: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:23.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:55:23 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:55:23.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:23 smithi100 bash[18597]: cluster 2024-09-10T18:55:22.328827+0000 mgr.y (mgr.24512) 799 : cluster [DBG] pgmap v1317: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:23.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:23 smithi100 bash[18597]: audit 2024-09-10T18:55:23.076990+0000 mgr.y (mgr.24512) 800 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:23 smithi100 bash[31607]: cluster 2024-09-10T18:55:22.328827+0000 mgr.y (mgr.24512) 799 : cluster [DBG] pgmap v1317: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:23.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:23 smithi100 bash[31607]: audit 2024-09-10T18:55:23.076990+0000 mgr.y (mgr.24512) 800 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:23.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:23 smithi151 bash[22565]: cluster 2024-09-10T18:55:22.328827+0000 mgr.y (mgr.24512) 799 : cluster [DBG] pgmap v1317: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:23.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:23 smithi151 bash[22565]: audit 2024-09-10T18:55:23.076990+0000 mgr.y (mgr.24512) 800 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:24.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:24 smithi100 bash[18597]: audit 2024-09-10T18:55:24.347993+0000 mon.c (mon.2) 561 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:24.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:24 smithi100 bash[31607]: audit 2024-09-10T18:55:24.347993+0000 mon.c (mon.2) 561 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:24.920 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:55:24 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:24.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:24 smithi151 bash[22565]: audit 2024-09-10T18:55:24.347993+0000 mon.c (mon.2) 561 : audit [DBG] from='mgr.24512 172.21.15.100:0/4107607224' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:25.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:25 smithi100 bash[18597]: cluster 2024-09-10T18:55:24.329643+0000 mgr.y (mgr.24512) 801 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:25.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:25 smithi100 bash[31607]: cluster 2024-09-10T18:55:24.329643+0000 mgr.y (mgr.24512) 801 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:25.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:25 smithi151 bash[22565]: cluster 2024-09-10T18:55:24.329643+0000 mgr.y (mgr.24512) 801 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:27.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:27 smithi100 bash[18597]: cluster 2024-09-10T18:55:26.331670+0000 mgr.y (mgr.24512) 802 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:27.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:27 smithi100 bash[31607]: cluster 2024-09-10T18:55:26.331670+0000 mgr.y (mgr.24512) 802 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:27.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:27 smithi151 bash[22565]: cluster 2024-09-10T18:55:26.331670+0000 mgr.y (mgr.24512) 802 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:29.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:29 smithi100 bash[18597]: cluster 2024-09-10T18:55:28.332497+0000 mgr.y (mgr.24512) 803 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:29.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:29 smithi100 bash[31607]: cluster 2024-09-10T18:55:28.332497+0000 mgr.y (mgr.24512) 803 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:29.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:29 smithi151 bash[22565]: cluster 2024-09-10T18:55:28.332497+0000 mgr.y (mgr.24512) 803 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:31.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:31 smithi100 bash[18597]: cluster 2024-09-10T18:55:30.333285+0000 mgr.y (mgr.24512) 804 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:31.794 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:31 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:31] "GET /metrics HTTP/1.1" 200 228772 "" "Prometheus/2.43.0" 2024-09-10T18:55:31.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:31 smithi100 bash[31607]: cluster 2024-09-10T18:55:30.333285+0000 mgr.y (mgr.24512) 804 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:31.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:31 smithi151 bash[22565]: cluster 2024-09-10T18:55:30.333285+0000 mgr.y (mgr.24512) 804 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:33.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:55:33 smithi151 bash[54154]: debug there is no tcmu-runner data available 2024-09-10T18:55:33.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:33 smithi100 bash[18597]: cluster 2024-09-10T18:55:32.335253+0000 mgr.y (mgr.24512) 805 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:33.879 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:33 smithi100 bash[18597]: audit 2024-09-10T18:55:33.088337+0000 mgr.y (mgr.24512) 806 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:33.879 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:33 smithi100 bash[31607]: cluster 2024-09-10T18:55:32.335253+0000 mgr.y (mgr.24512) 805 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:33.880 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:33 smithi100 bash[31607]: audit 2024-09-10T18:55:33.088337+0000 mgr.y (mgr.24512) 806 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:33.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:33 smithi151 bash[22565]: cluster 2024-09-10T18:55:32.335253+0000 mgr.y (mgr.24512) 805 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:33.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:33 smithi151 bash[22565]: audit 2024-09-10T18:55:33.088337+0000 mgr.y (mgr.24512) 806 : audit [DBG] from='client.24485 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:34.920 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:55:34 smithi151 bash[23560]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:35.453 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool delete 3713687d-818c-43e2-97c0-4c4d90a80f33 3713687d-818c-43e2-97c0-4c4d90a80f33 --yes-i-really-really-mean-it 2024-09-10T18:55:35.564 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 -- 172.21.15.100:0/1768461473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fbbe0 msgr2=0x7f3a4c0fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:35.564 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 --2- 172.21.15.100:0/1768461473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fbbe0 0x7f3a4c0fbfb0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0049c0 tx=0x7f3a3c009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:35.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 -- 172.21.15.100:0/1768461473 shutdown_connections 2024-09-10T18:55:35.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 --2- 172.21.15.100:0/1768461473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4c1011c0 0x7f3a4c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:35.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 --2- 172.21.15.100:0/1768461473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fc580 0x7f3a4c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:35.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 --2- 172.21.15.100:0/1768461473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fbbe0 0x7f3a4c0fbfb0 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:35.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 -- 172.21.15.100:0/1768461473 >> 172.21.15.100:0/1768461473 conn(0x7f3a4c071930 msgr2=0x7f3a4c0f9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:35.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 -- 172.21.15.100:0/1768461473 shutdown_connections 2024-09-10T18:55:35.565 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.560+0000 7f3a5203d700 1 -- 172.21.15.100:0/1768461473 wait complete. 2024-09-10T18:55:35.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 Processor -- start 2024-09-10T18:55:35.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 -- start start 2024-09-10T18:55:35.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fbbe0 0x7f3a4c119760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:35.567 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fc580 0x7f3a4c119ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4c1011c0 0x7f3a4c1139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c113fd0 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c114140 con 0x7f3a4c1011c0 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c114300 con 0x7f3a4c0fc580 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fbbe0 0x7f3a4c119760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fbbe0 0x7f3a4c119760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57102/0 (socket says 172.21.15.100:57102) 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 -- 172.21.15.100:0/1731796956 learned_addr learned my addr 172.21.15.100:0/1731796956 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a43fff700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fc580 0x7f3a4c119ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4bfff700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4c1011c0 0x7f3a4c1139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 -- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fc580 msgr2=0x7f3a4c119ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:35.568 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fc580 0x7f3a4c119ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 -- 172.21.15.100:0/1731796956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4c1011c0 msgr2=0x7f3a4c1139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4c1011c0 0x7f3a4c1139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c005040 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4bfff700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4c1011c0 0x7f3a4c1139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a4b7fe700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fbbe0 0x7f3a4c119760 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b0a0 tx=0x7f3a3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00e050 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f3a3c009820 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c114580 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.569 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c01bba0 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.570 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a5203d700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c120790 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.564+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a300050b0 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.568+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f3a3c0093b0 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.568+0000 7f3a48ff9700 1 --2- 172.21.15.100:0/1731796956 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3a3806f010 0x7f3a380714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:35.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.568+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(772..772 src has 1..772) v4 ==== 8175+0+0 (secure 0 0 0) 0x7f3a3c090500 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.568+0000 7f3a43fff700 1 --2- 172.21.15.100:0/1731796956 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3a3806f010 0x7f3a380714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:35.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.568+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=773}) v3 -- 0x7f3a3807ab60 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.571 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.568+0000 7f3a43fff700 1 --2- 172.21.15.100:0/1731796956 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3a3806f010 0x7f3a380714c0 secure :-1 s=READY pgs=3553 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009b80 tx=0x7f3a3400b040 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:35.578 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.576+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3a3c05cb70 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.752 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:35.748+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a30005360 con 0x7f3a4c0fbbe0 2024-09-10T18:55:35.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:35 smithi100 bash[18597]: cluster 2024-09-10T18:55:34.336076+0000 mgr.y (mgr.24512) 807 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:35.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:35 smithi100 bash[31607]: cluster 2024-09-10T18:55:34.336076+0000 mgr.y (mgr.24512) 807 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:35.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:35 smithi151 bash[22565]: cluster 2024-09-10T18:55:34.336076+0000 mgr.y (mgr.24512) 807 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:36.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.520+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 7 ==== osd_map(773..773 src has 1..773) v4 ==== 288+0+0 (secure 0 0 0) 0x7f3a3c009660 con 0x7f3a4c0fbbe0 2024-09-10T18:55:36.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.520+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7f3a3807b490 con 0x7f3a4c0fbbe0 2024-09-10T18:55:36.528 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.524+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]=0 pool '3713687d-818c-43e2-97c0-4c4d90a80f33' removed v773) v1 ==== 248+0+0 (secure 0 0 0) 0x7f3a3c01f100 con 0x7f3a4c0fbbe0 2024-09-10T18:55:36.639 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.636+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a300058b0 con 0x7f3a4c0fbbe0 2024-09-10T18:55:36.640 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.636+0000 7f3a48ff9700 1 -- 172.21.15.100:0/1731796956 <== mon.0 v2:172.21.15.100:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]=0 pool '3713687d-818c-43e2-97c0-4c4d90a80f33' does not exist v773) v1 ==== 255+0+0 (secure 0 0 0) 0x7f3a3c029020 con 0x7f3a4c0fbbe0 2024-09-10T18:55:36.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3a3806f010 msgr2=0x7f3a380714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:36.642 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 --2- 172.21.15.100:0/1731796956 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3a3806f010 0x7f3a380714c0 secure :-1 s=READY pgs=3553 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009b80 tx=0x7f3a3400b040 comp rx=0 tx=0).stop 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fbbe0 msgr2=0x7f3a4c119760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fbbe0 0x7f3a4c119760 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b0a0 tx=0x7f3a3c009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 shutdown_connections 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 --2- 172.21.15.100:0/1731796956 >> v2:172.21.15.100:6800/2155730476 conn(0x7f3a3806f010 0x7f3a380714c0 unknown :-1 s=CLOSED pgs=3553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a4c1011c0 0x7f3a4c1139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a4c0fc580 0x7f3a4c119ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 --2- 172.21.15.100:0/1731796956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a4c0fbbe0 0x7f3a4c119760 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 >> 172.21.15.100:0/1731796956 conn(0x7f3a4c071930 msgr2=0x7f3a4c072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 shutdown_connections 2024-09-10T18:55:36.643 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.640+0000 7f3a41ffb700 1 -- 172.21.15.100:0/1731796956 wait complete. 2024-09-10T18:55:36.644 INFO:tasks.workunit.client.0.smithi100.stderr:pool '3713687d-818c-43e2-97c0-4c4d90a80f33' does not exist 2024-09-10T18:55:36.651 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool delete a36ef4a4-357c-465b-9b7d-4dcd9b201084 a36ef4a4-357c-465b-9b7d-4dcd9b201084 --yes-i-really-really-mean-it 2024-09-10T18:55:36.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.752+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2559415860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10103710 msgr2=0x7f8d100fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:36.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.752+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2559415860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10103710 0x7f8d100fcfd0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0049c0 tx=0x7f8d0c009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:36.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.752+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2559415860 shutdown_connections 2024-09-10T18:55:36.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.752+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2559415860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 0x7f8d10109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.752+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2559415860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10103710 0x7f8d100fcfd0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.757 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.752+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2559415860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10102e00 0x7f8d101031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.752+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2559415860 >> 172.21.15.100:0/2559415860 conn(0x7f8d10073e30 msgr2=0x7f8d10074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:36.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2559415860 shutdown_connections 2024-09-10T18:55:36.758 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2559415860 wait complete. 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 Processor -- start 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- start start 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 0x7f8d10113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10102e00 0x7f8d10113750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10103710 0x7f8d1010d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1010d9f0 con 0x7f8d10102e00 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1010db60 con 0x7f8d10103710 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1010dd20 con 0x7f8d100fd700 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 0x7f8d10113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:36.760 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 0x7f8d10113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:36838/0 (socket says 172.21.15.100:36838) 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 -- 172.21.15.100:0/2656739642 learned_addr learned my addr 172.21.15.100:0/2656739642 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d154a6700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10102e00 0x7f8d10113750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 -- 172.21.15.100:0/2656739642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10103710 msgr2=0x7f8d1010d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d164a8700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10103710 0x7f8d1010d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10103710 0x7f8d1010d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 -- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10102e00 msgr2=0x7f8d10113750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10102e00 0x7f8d10113750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c005040 con 0x7f8d100fd700 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d154a6700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10102e00 0x7f8d10113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d164a8700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10103710 0x7f8d1010d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d15ca7700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 0x7f8d10113210 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8d00002b00 tx=0x7f8d000046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00007cc0 con 0x7f8d100fd700 2024-09-10T18:55:36.761 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1019+0+0 (secure 0 0 0) 0x7f8d0000f040 con 0x7f8d100fd700 2024-09-10T18:55:36.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1010e000 con 0x7f8d100fd700 2024-09-10T18:55:36.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d000092e0 con 0x7f8d100fd700 2024-09-10T18:55:36.762 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.756+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1011a200 con 0x7f8d100fd700 2024-09-10T18:55:36.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.760+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d100fe550 con 0x7f8d100fd700 2024-09-10T18:55:36.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.760+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 21) v1 ==== 90804+0+0 (secure 0 0 0) 0x7f8d00009750 con 0x7f8d100fd700 2024-09-10T18:55:36.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.760+0000 7f8d067fc700 1 --2- 172.21.15.100:0/2656739642 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8cfc06f010 0x7f8cfc0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:55:36.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.760+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(773..773 src has 1..773) v4 ==== 7787+0+0 (secure 0 0 0) 0x7f8d0008e8e0 con 0x7f8d100fd700 2024-09-10T18:55:36.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.760+0000 7f8d154a6700 1 --2- 172.21.15.100:0/2656739642 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8cfc06f010 0x7f8cfc0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:55:36.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.760+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7f8cfc07a2c0 con 0x7f8d100fd700 2024-09-10T18:55:36.763 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.760+0000 7f8d154a6700 1 --2- 172.21.15.100:0/2656739642 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8cfc06f010 0x7f8cfc0714c0 secure :-1 s=READY pgs=3554 cs=0 l=1 rev1=1 crypto rx=0x7f8d1010ead0 tx=0x7f8d0c009e30 comp rx=0 tx=0).ready entity=mgr.24512 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:55:36.768 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.764+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8d0005b0f0 con 0x7f8d100fd700 2024-09-10T18:55:36.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:36 smithi100 bash[18597]: audit 2024-09-10T18:55:35.753090+0000 mon.a (mon.0) 3615 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:36.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:36 smithi100 bash[31607]: audit 2024-09-10T18:55:35.753090+0000 mon.a (mon.0) 3615 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:36.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:36 smithi151 bash[22565]: audit 2024-09-10T18:55:35.753090+0000 mon.a (mon.0) 3615 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:36.947 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:36.944+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d1010c290 con 0x7f8d100fd700 2024-09-10T18:55:37.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.532+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(774..774 src has 1..774) v4 ==== 288+0+0 (secure 0 0 0) 0x7f8d0005ae30 con 0x7f8d100fd700 2024-09-10T18:55:37.536 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.532+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=775}) v3 -- 0x7f8cfc07abb0 con 0x7f8d100fd700 2024-09-10T18:55:37.546 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.544+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]=0 pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' removed v774) v1 ==== 248+0+0 (secure 0 0 0) 0x7f8d0005aa90 con 0x7f8d100fd700 2024-09-10T18:55:37.651 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.648+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d1010c260 con 0x7f8d100fd700 2024-09-10T18:55:37.653 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d067fc700 1 -- 172.21.15.100:0/2656739642 <== mon.2 v2:172.21.15.100:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]=0 pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' does not exist v774) v1 ==== 255+0+0 (secure 0 0 0) 0x7f8d0005ac70 con 0x7f8d100fd700 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8cfc06f010 msgr2=0x7f8cfc0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2656739642 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8cfc06f010 0x7f8cfc0714c0 secure :-1 s=READY pgs=3554 cs=0 l=1 rev1=1 crypto rx=0x7f8d1010ead0 tx=0x7f8d0c009e30 comp rx=0 tx=0).stop 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 msgr2=0x7f8d10113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 0x7f8d10113210 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8d00002b00 tx=0x7f8d000046b0 comp rx=0 tx=0).stop 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 shutdown_connections 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2656739642 >> v2:172.21.15.100:6800/2155730476 conn(0x7f8cfc06f010 0x7f8cfc0714c0 unknown :-1 s=CLOSED pgs=3554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d10103710 0x7f8d1010d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d10102e00 0x7f8d10113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:37.656 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 --2- 172.21.15.100:0/2656739642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d100fd700 0x7f8d10113210 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:55:37.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 >> 172.21.15.100:0/2656739642 conn(0x7f8d10073e30 msgr2=0x7f8d101009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:37.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 shutdown_connections 2024-09-10T18:55:37.657 INFO:tasks.workunit.client.0.smithi100.stderr:2024-09-10T18:55:37.652+0000 7f8d17f0b700 1 -- 172.21.15.100:0/2656739642 wait complete. 2024-09-10T18:55:37.657 INFO:tasks.workunit.client.0.smithi100.stderr:pool 'a36ef4a4-357c-465b-9b7d-4dcd9b201084' does not exist 2024-09-10T18:55:37.664 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo OK 2024-09-10T18:55:37.665 INFO:tasks.workunit.client.0.smithi100.stdout:OK 2024-09-10T18:55:37.666 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T18:55:37.666 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T18:55:37.675 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-10T18:55:37.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T18:55:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[18597]: cluster 2024-09-10T18:55:36.338055+0000 mgr.y (mgr.24512) 808 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[18597]: audit 2024-09-10T18:55:36.527930+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:55:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[18597]: cluster 2024-09-10T18:55:36.528083+0000 mon.a (mon.0) 3617 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-09-10T18:55:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[18597]: audit 2024-09-10T18:55:36.640421+0000 mon.a (mon.0) 3618 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[18597]: audit 2024-09-10T18:55:36.947768+0000 mon.c (mon.2) 562 : audit [INF] from='client.? 172.21.15.100:0/2656739642' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.794 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[18597]: audit 2024-09-10T18:55:36.948407+0000 mon.a (mon.0) 3619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[31607]: cluster 2024-09-10T18:55:36.338055+0000 mgr.y (mgr.24512) 808 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:37.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[31607]: audit 2024-09-10T18:55:36.527930+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:55:37.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[31607]: cluster 2024-09-10T18:55:36.528083+0000 mon.a (mon.0) 3617 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-09-10T18:55:37.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[31607]: audit 2024-09-10T18:55:36.640421+0000 mon.a (mon.0) 3618 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[31607]: audit 2024-09-10T18:55:36.947768+0000 mon.c (mon.2) 562 : audit [INF] from='client.? 172.21.15.100:0/2656739642' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.795 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:37 smithi100 bash[31607]: audit 2024-09-10T18:55:36.948407+0000 mon.a (mon.0) 3619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:37 smithi151 bash[22565]: cluster 2024-09-10T18:55:36.338055+0000 mgr.y (mgr.24512) 808 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 900 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:37.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:37 smithi151 bash[22565]: audit 2024-09-10T18:55:36.527930+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:55:37.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:37 smithi151 bash[22565]: cluster 2024-09-10T18:55:36.528083+0000 mon.a (mon.0) 3617 : cluster [DBG] osdmap e773: 8 total, 8 up, 8 in 2024-09-10T18:55:37.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:37 smithi151 bash[22565]: audit 2024-09-10T18:55:36.640421+0000 mon.a (mon.0) 3618 : audit [INF] from='client.? 172.21.15.100:0/1731796956' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "3713687d-818c-43e2-97c0-4c4d90a80f33", "pool2": "3713687d-818c-43e2-97c0-4c4d90a80f33", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:37 smithi151 bash[22565]: audit 2024-09-10T18:55:36.947768+0000 mon.c (mon.2) 562 : audit [INF] from='client.? 172.21.15.100:0/2656739642' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:37 smithi151 bash[22565]: audit 2024-09-10T18:55:36.948407+0000 mon.a (mon.0) 3619 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "pool2": "a36ef4a4-357c-465b-9b7d-4dcd9b201084", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:55:37.942 DEBUG:teuthology.parallel:result is None 2024-09-10T18:55:37.943 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:55:37.956 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:55:37.956 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:55:37.975 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:55:37.975 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T18:55:37.985 INFO:tasks.cephadm:Teardown begin 2024-09-10T18:55:37.986 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:55:38.031 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:55:38.048 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T18:55:38.048 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:55:38.055 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:55:38.095 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T18:55:38.095 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T18:55:38.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:55:38.293 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:38 smithi100 systemd[1]: Stopping Ceph mon.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:55:38.611 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:38 smithi100 bash[18597]: debug 2024-09-10T18:55:38.316+0000 7f6cd882e640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-09-10T18:55:38.611 INFO:journalctl@ceph.mon.a.smithi100.stdout:Sep 10 18:55:38 smithi100 bash[18597]: debug 2024-09-10T18:55:38.316+0000 7f6cd882e640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T18:55:39.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T18:55:39.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:55:39.292 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T18:55:39.292 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T18:55:39.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.c 2024-09-10T18:55:39.445 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:39 smithi100 systemd[1]: Stopping Ceph mon.c for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:55:39.793 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:39 smithi100 bash[18863]: [10/Sep/2024:18:55:39] ENGINE Bus STOPPING 2024-09-10T18:55:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:39 smithi100 bash[31607]: debug 2024-09-10T18:55:39.440+0000 7fd280803640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-09-10T18:55:39.794 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:39 smithi100 bash[31607]: debug 2024-09-10T18:55:39.440+0000 7fd280803640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:55:40.147 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:39 smithi100 bash[18863]: [10/Sep/2024:18:55:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:55:40.147 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:39 smithi100 bash[18863]: [10/Sep/2024:18:55:39] ENGINE Bus STOPPED 2024-09-10T18:55:40.147 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:39 smithi100 bash[18863]: [10/Sep/2024:18:55:39] ENGINE Bus STARTING 2024-09-10T18:55:40.147 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:39 smithi100 bash[18863]: [10/Sep/2024:18:55:39] ENGINE Serving on http://:::9283 2024-09-10T18:55:40.526 INFO:journalctl@ceph.mon.c.smithi100.stdout:Sep 10 18:55:40 smithi100 bash[140100]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-mon-c 2024-09-10T18:55:40.526 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:40 smithi100 bash[18863]: [10/Sep/2024:18:55:39] ENGINE Bus STARTED 2024-09-10T18:55:40.541 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T18:55:40.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:55:40.570 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T18:55:40.570 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T18:55:40.570 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.b 2024-09-10T18:55:40.920 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:40 smithi151 systemd[1]: Stopping Ceph mon.b for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:55:40.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:40 smithi151 bash[22565]: debug 2024-09-10T18:55:40.716+0000 7f69737e6640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-09-10T18:55:40.921 INFO:journalctl@ceph.mon.b.smithi151.stdout:Sep 10 18:55:40 smithi151 bash[22565]: debug 2024-09-10T18:55:40.716+0000 7f69737e6640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:55:41.727 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T18:55:41.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:55:41.778 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T18:55:41.778 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T18:55:41.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:55:41.792 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:41 smithi100 bash[18863]: ::ffff:172.21.15.151 - - [10/Sep/2024:18:55:41] "GET /metrics HTTP/1.1" 200 228772 "" "Prometheus/2.43.0" 2024-09-10T18:55:42.043 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:41 smithi100 systemd[1]: Stopping Ceph mgr.y for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:55:42.043 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:41 smithi100 bash[18863]: debug 2024-09-10T18:55:41.936+0000 7f188ad42640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:55:42.044 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Sep 10 18:55:41 smithi100 bash[18863]: debug 2024-09-10T18:55:41.936+0000 7f188ad42640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T18:55:42.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T18:55:42.839 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:55:42.840 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T18:55:42.840 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T18:55:42.840 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.x 2024-09-10T18:55:43.170 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Sep 10 18:55:42 smithi151 systemd[1]: Stopping Ceph mgr.x for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:55:43.723 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T18:55:43.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:55:43.755 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T18:55:43.755 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T18:55:43.755 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.0 2024-09-10T18:55:44.044 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:55:43 smithi100 systemd[1]: Stopping Ceph osd.0 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:55:44.044 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:55:43 smithi100 bash[34936]: debug 2024-09-10T18:55:43.916+0000 7fc4e0d75640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:55:44.044 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:55:43 smithi100 bash[34936]: debug 2024-09-10T18:55:43.916+0000 7fc4e0d75640 -1 osd.0 774 *** Got signal Terminated *** 2024-09-10T18:55:44.044 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:55:43 smithi100 bash[34936]: debug 2024-09-10T18:55:43.916+0000 7fc4e0d75640 -1 osd.0 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:55:50.284 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:55:49 smithi100 bash[140317]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-0 2024-09-10T18:55:50.284 INFO:journalctl@ceph.osd.0.smithi100.stdout:Sep 10 18:55:50 smithi100 bash[140391]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-0 2024-09-10T18:55:52.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T18:55:52.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:55:52.389 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T18:55:52.389 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T18:55:52.389 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.1 2024-09-10T18:55:52.794 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 10 18:55:52 smithi100 systemd[1]: Stopping Ceph osd.1 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:55:52.794 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 10 18:55:52 smithi100 bash[42533]: debug 2024-09-10T18:55:52.599+0000 7ff7d2db8640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:55:52.794 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 10 18:55:52 smithi100 bash[42533]: debug 2024-09-10T18:55:52.599+0000 7ff7d2db8640 -1 osd.1 774 *** Got signal Terminated *** 2024-09-10T18:55:52.794 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 10 18:55:52 smithi100 bash[42533]: debug 2024-09-10T18:55:52.599+0000 7ff7d2db8640 -1 osd.1 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:55:59.044 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 10 18:55:58 smithi100 bash[140531]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-1 2024-09-10T18:55:59.044 INFO:journalctl@ceph.osd.1.smithi100.stdout:Sep 10 18:55:58 smithi100 bash[140603]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-1 2024-09-10T18:56:00.438 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:00 smithi100 bash[68068]: ts=2024-09-10T18:56:00.159Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:00.438 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:00 smithi100 bash[68068]: ts=2024-09-10T18:56:00.160Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:56:01.140 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T18:56:01.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:56:01.174 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T18:56:01.174 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T18:56:01.174 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.2 2024-09-10T18:56:01.544 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 10 18:56:01 smithi100 systemd[1]: Stopping Ceph osd.2 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:56:01.544 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 10 18:56:01 smithi100 bash[50329]: debug 2024-09-10T18:56:01.319+0000 7f4a0c43c640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:56:01.544 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 10 18:56:01 smithi100 bash[50329]: debug 2024-09-10T18:56:01.319+0000 7f4a0c43c640 -1 osd.2 774 *** Got signal Terminated *** 2024-09-10T18:56:01.544 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 10 18:56:01 smithi100 bash[50329]: debug 2024-09-10T18:56:01.319+0000 7f4a0c43c640 -1 osd.2 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:56:07.711 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 10 18:56:07 smithi100 bash[140741]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-2 2024-09-10T18:56:07.711 INFO:journalctl@ceph.osd.2.smithi100.stdout:Sep 10 18:56:07 smithi100 bash[140815]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-2 2024-09-10T18:56:09.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T18:56:09.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:56:09.709 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T18:56:09.709 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T18:56:09.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.3 2024-09-10T18:56:10.044 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:56:09 smithi100 systemd[1]: Stopping Ceph osd.3 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:56:10.044 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:56:09 smithi100 bash[58092]: debug 2024-09-10T18:56:09.915+0000 7fabeeff8640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:56:10.044 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:56:09 smithi100 bash[58092]: debug 2024-09-10T18:56:09.915+0000 7fabeeff8640 -1 osd.3 774 *** Got signal Terminated *** 2024-09-10T18:56:10.044 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:56:09 smithi100 bash[58092]: debug 2024-09-10T18:56:09.915+0000 7fabeeff8640 -1 osd.3 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:56:10.407 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:10 smithi151 bash[47658]: debug 2024-09-10T18:56:10.137+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:10.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:10 smithi100 bash[68068]: ts=2024-09-10T18:56:10.159Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:10.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:10 smithi100 bash[68068]: ts=2024-09-10T18:56:10.160Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:10.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:10 smithi100 bash[68068]: ts=2024-09-10T18:56:10.162Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:56:10.670 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:10 smithi151 bash[26435]: debug 2024-09-10T18:56:10.397+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:11.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:11 smithi151 bash[47658]: debug 2024-09-10T18:56:11.181+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:11.670 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:11 smithi151 bash[26435]: debug 2024-09-10T18:56:11.385+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:12.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:12 smithi151 bash[47658]: debug 2024-09-10T18:56:12.141+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:12.420 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:12 smithi151 bash[26435]: debug 2024-09-10T18:56:12.369+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:13.496 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:13 smithi151 bash[26435]: debug 2024-09-10T18:56:13.345+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:13.496 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:13 smithi151 bash[47658]: debug 2024-09-10T18:56:13.165+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:13.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:13 smithi151 bash[33542]: debug 2024-09-10T18:56:13.485+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:14.516 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:14 smithi151 bash[26435]: debug 2024-09-10T18:56:14.321+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:14.516 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:14 smithi151 bash[47658]: debug 2024-09-10T18:56:14.177+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:14.543 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:56:14 smithi100 bash[58092]: debug 2024-09-10T18:56:14.167+0000 7fabe860a640 -1 osd.3 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.455719+0000 front 2024-09-10T18:55:47.455607+0000 (oldest deadline 2024-09-10T18:56:13.856465+0000) 2024-09-10T18:56:14.919 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:14 smithi151 bash[33542]: debug 2024-09-10T18:56:14.505+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:14.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:14 smithi151 bash[40568]: debug 2024-09-10T18:56:14.537+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:15.503 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:15 smithi151 bash[26435]: debug 2024-09-10T18:56:15.305+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:15.503 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:15 smithi151 bash[47658]: debug 2024-09-10T18:56:15.177+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:15.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:15 smithi151 bash[40568]: debug 2024-09-10T18:56:15.493+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:15.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:15 smithi151 bash[33542]: debug 2024-09-10T18:56:15.545+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:16.293 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:56:16 smithi100 bash[140956]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-3 2024-09-10T18:56:16.294 INFO:journalctl@ceph.osd.3.smithi100.stdout:Sep 10 18:56:16 smithi100 bash[141031]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-3 2024-09-10T18:56:16.529 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:16 smithi151 bash[47658]: debug 2024-09-10T18:56:16.165+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:16.529 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:16 smithi151 bash[26435]: debug 2024-09-10T18:56:16.281+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:16.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:16 smithi151 bash[33542]: debug 2024-09-10T18:56:16.569+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:16.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:16 smithi151 bash[40568]: debug 2024-09-10T18:56:16.521+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:17.419 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:17 smithi151 bash[26435]: debug 2024-09-10T18:56:17.261+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:17.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:17 smithi151 bash[47658]: debug 2024-09-10T18:56:17.153+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:17.919 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:17 smithi151 bash[40568]: debug 2024-09-10T18:56:17.517+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:17.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:17 smithi151 bash[33542]: debug 2024-09-10T18:56:17.561+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:18.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T18:56:18.264 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:56:18.264 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T18:56:18.264 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T18:56:18.265 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.4 2024-09-10T18:56:18.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[47658]: debug 2024-09-10T18:56:18.133+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:18.420 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:18 smithi151 systemd[1]: Stopping Ceph osd.4 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:56:18.420 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[26435]: debug 2024-09-10T18:56:18.277+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:18.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[40568]: debug 2024-09-10T18:56:18.517+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:18.920 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[26435]: debug 2024-09-10T18:56:18.477+0000 7fc3a2fff640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:56:18.920 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[26435]: debug 2024-09-10T18:56:18.477+0000 7fc3a2fff640 -1 osd.4 774 *** Got signal Terminated *** 2024-09-10T18:56:18.920 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[26435]: debug 2024-09-10T18:56:18.477+0000 7fc3a2fff640 -1 osd.4 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:56:18.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[33542]: debug 2024-09-10T18:56:18.609+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:18.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:18 smithi151 bash[33542]: debug 2024-09-10T18:56:18.609+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:19.420 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:19 smithi151 bash[26435]: debug 2024-09-10T18:56:19.277+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:19.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:19 smithi151 bash[47658]: debug 2024-09-10T18:56:19.109+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:19.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:19 smithi151 bash[40568]: debug 2024-09-10T18:56:19.513+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:19.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:19 smithi151 bash[33542]: debug 2024-09-10T18:56:19.565+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:19.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:19 smithi151 bash[33542]: debug 2024-09-10T18:56:19.565+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:20.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:20 smithi151 bash[47658]: debug 2024-09-10T18:56:20.125+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:20.420 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:20 smithi151 bash[26435]: debug 2024-09-10T18:56:20.313+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:20.420 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:20 smithi151 bash[26435]: debug 2024-09-10T18:56:20.313+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.333151+0000 front 2024-09-10T18:55:57.333219+0000 (oldest deadline 2024-09-10T18:56:20.233072+0000) 2024-09-10T18:56:20.543 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:20 smithi100 bash[68068]: ts=2024-09-10T18:56:20.160Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:20.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:20 smithi100 bash[68068]: ts=2024-09-10T18:56:20.161Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:20.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:20 smithi100 bash[68068]: ts=2024-09-10T18:56:20.162Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:56:20.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:20 smithi151 bash[40568]: debug 2024-09-10T18:56:20.545+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:20.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:20 smithi151 bash[33542]: debug 2024-09-10T18:56:20.545+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:20.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:20 smithi151 bash[33542]: debug 2024-09-10T18:56:20.545+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:21.516 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:21 smithi151 bash[47658]: debug 2024-09-10T18:56:21.173+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:21.516 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:21 smithi151 bash[26435]: debug 2024-09-10T18:56:21.337+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:21.516 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:21 smithi151 bash[26435]: debug 2024-09-10T18:56:21.337+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.333151+0000 front 2024-09-10T18:55:57.333219+0000 (oldest deadline 2024-09-10T18:56:20.233072+0000) 2024-09-10T18:56:21.921 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:21 smithi151 bash[33542]: debug 2024-09-10T18:56:21.505+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:21.921 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:21 smithi151 bash[33542]: debug 2024-09-10T18:56:21.505+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:21.921 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:21 smithi151 bash[40568]: debug 2024-09-10T18:56:21.557+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:22.555 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:22 smithi151 bash[47658]: debug 2024-09-10T18:56:22.197+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:22.555 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:22 smithi151 bash[26435]: debug 2024-09-10T18:56:22.313+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:22.555 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:22 smithi151 bash[26435]: debug 2024-09-10T18:56:22.313+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.333151+0000 front 2024-09-10T18:55:57.333219+0000 (oldest deadline 2024-09-10T18:56:20.233072+0000) 2024-09-10T18:56:22.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:22 smithi151 bash[33542]: debug 2024-09-10T18:56:22.545+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:22.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:22 smithi151 bash[33542]: debug 2024-09-10T18:56:22.545+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:22.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:22 smithi151 bash[40568]: debug 2024-09-10T18:56:22.605+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:22.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:22 smithi151 bash[40568]: debug 2024-09-10T18:56:22.605+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:23.561 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[47658]: debug 2024-09-10T18:56:23.201+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:23.561 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[47658]: debug 2024-09-10T18:56:23.201+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:23.561 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[26435]: debug 2024-09-10T18:56:23.289+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:46.831501+0000 front 2024-09-10T18:55:46.831584+0000 (oldest deadline 2024-09-10T18:56:09.731229+0000) 2024-09-10T18:56:23.561 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[26435]: debug 2024-09-10T18:56:23.289+0000 7fc39ee16640 -1 osd.4 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.333151+0000 front 2024-09-10T18:55:57.333219+0000 (oldest deadline 2024-09-10T18:56:20.233072+0000) 2024-09-10T18:56:23.919 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[33542]: debug 2024-09-10T18:56:23.549+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:23.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[33542]: debug 2024-09-10T18:56:23.549+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:23.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[40568]: debug 2024-09-10T18:56:23.565+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:23.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:23 smithi151 bash[40568]: debug 2024-09-10T18:56:23.565+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:24.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[47658]: debug 2024-09-10T18:56:24.209+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:24.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[47658]: debug 2024-09-10T18:56:24.209+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:24.844 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[59372]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-4 2024-09-10T18:56:24.845 INFO:journalctl@ceph.osd.4.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[59446]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-4 2024-09-10T18:56:24.845 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[40568]: debug 2024-09-10T18:56:24.601+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:24.845 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[40568]: debug 2024-09-10T18:56:24.601+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:24.845 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[33542]: debug 2024-09-10T18:56:24.581+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:24.845 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:24 smithi151 bash[33542]: debug 2024-09-10T18:56:24.581+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:25.586 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:25 smithi151 bash[47658]: debug 2024-09-10T18:56:25.201+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:25.586 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:25 smithi151 bash[47658]: debug 2024-09-10T18:56:25.201+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:25.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:25 smithi151 bash[33542]: debug 2024-09-10T18:56:25.601+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:25.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:25 smithi151 bash[33542]: debug 2024-09-10T18:56:25.601+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:25.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:25 smithi151 bash[40568]: debug 2024-09-10T18:56:25.577+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:25.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:25 smithi151 bash[40568]: debug 2024-09-10T18:56:25.577+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:26.501 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:26 smithi151 bash[47658]: debug 2024-09-10T18:56:26.233+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:26.501 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:26 smithi151 bash[47658]: debug 2024-09-10T18:56:26.233+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:26.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:26 smithi151 bash[40568]: debug 2024-09-10T18:56:26.565+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:26.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:26 smithi151 bash[40568]: debug 2024-09-10T18:56:26.565+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:26.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:26 smithi151 bash[33542]: debug 2024-09-10T18:56:26.633+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:26.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:26 smithi151 bash[33542]: debug 2024-09-10T18:56:26.633+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:27.013 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T18:56:27.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:56:27.050 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T18:56:27.050 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T18:56:27.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.5 2024-09-10T18:56:27.397 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[47658]: debug 2024-09-10T18:56:27.241+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:27.397 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[47658]: debug 2024-09-10T18:56:27.241+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:27.397 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:27 smithi151 systemd[1]: Stopping Ceph osd.5 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:56:27.397 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[33542]: debug 2024-09-10T18:56:27.237+0000 7f3454dfd640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:56:27.397 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[33542]: debug 2024-09-10T18:56:27.237+0000 7f3454dfd640 -1 osd.5 774 *** Got signal Terminated *** 2024-09-10T18:56:27.397 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[33542]: debug 2024-09-10T18:56:27.237+0000 7f3454dfd640 -1 osd.5 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:56:27.661 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[57256]: logger=cleanup t=2024-09-10T18:56:27.389582202Z level=info msg="Completed cleanup jobs" duration=91.988585ms 2024-09-10T18:56:27.661 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[40568]: debug 2024-09-10T18:56:27.553+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:27.662 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[40568]: debug 2024-09-10T18:56:27.553+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:27.919 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[33542]: debug 2024-09-10T18:56:27.653+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:27.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[33542]: debug 2024-09-10T18:56:27.653+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:27.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:27 smithi151 bash[33542]: debug 2024-09-10T18:56:27.653+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:03.837223+0000 front 2024-09-10T18:56:03.837244+0000 (oldest deadline 2024-09-10T18:56:27.237431+0000) 2024-09-10T18:56:28.611 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:28 smithi151 bash[47658]: debug 2024-09-10T18:56:28.281+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:28.612 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:28 smithi151 bash[47658]: debug 2024-09-10T18:56:28.281+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:28.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:28 smithi151 bash[40568]: debug 2024-09-10T18:56:28.601+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:28.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:28 smithi151 bash[40568]: debug 2024-09-10T18:56:28.601+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:28.921 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:28 smithi151 bash[33542]: debug 2024-09-10T18:56:28.665+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:28.921 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:28 smithi151 bash[33542]: debug 2024-09-10T18:56:28.665+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:28.921 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:28 smithi151 bash[33542]: debug 2024-09-10T18:56:28.665+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:03.837223+0000 front 2024-09-10T18:56:03.837244+0000 (oldest deadline 2024-09-10T18:56:27.237431+0000) 2024-09-10T18:56:29.634 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[47658]: debug 2024-09-10T18:56:29.257+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:29.634 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[47658]: debug 2024-09-10T18:56:29.257+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:29.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[33542]: debug 2024-09-10T18:56:29.661+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:29.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[33542]: debug 2024-09-10T18:56:29.661+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:29.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[33542]: debug 2024-09-10T18:56:29.661+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:03.837223+0000 front 2024-09-10T18:56:03.837244+0000 (oldest deadline 2024-09-10T18:56:27.237431+0000) 2024-09-10T18:56:29.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[40568]: debug 2024-09-10T18:56:29.625+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:29.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[40568]: debug 2024-09-10T18:56:29.625+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:29.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:29 smithi151 bash[40568]: debug 2024-09-10T18:56:29.625+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:30.543 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:30 smithi100 bash[68068]: ts=2024-09-10T18:56:30.161Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:56:30.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:30 smithi100 bash[68068]: ts=2024-09-10T18:56:30.162Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:30.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:30 smithi100 bash[68068]: ts=2024-09-10T18:56:30.163Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:56:30.594 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[47658]: debug 2024-09-10T18:56:30.217+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:30.595 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[47658]: debug 2024-09-10T18:56:30.217+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:30.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[40568]: debug 2024-09-10T18:56:30.585+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:30.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[40568]: debug 2024-09-10T18:56:30.585+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:30.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[40568]: debug 2024-09-10T18:56:30.585+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:30.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[33542]: debug 2024-09-10T18:56:30.641+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:30.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[33542]: debug 2024-09-10T18:56:30.641+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:30.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:30 smithi151 bash[33542]: debug 2024-09-10T18:56:30.641+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:03.837223+0000 front 2024-09-10T18:56:03.837244+0000 (oldest deadline 2024-09-10T18:56:27.237431+0000) 2024-09-10T18:56:31.599 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[47658]: debug 2024-09-10T18:56:31.241+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:31.599 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[47658]: debug 2024-09-10T18:56:31.241+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:31.599 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[47658]: debug 2024-09-10T18:56:31.241+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:31.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[33542]: debug 2024-09-10T18:56:31.617+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:47.533761+0000 front 2024-09-10T18:55:47.533724+0000 (oldest deadline 2024-09-10T18:56:12.833247+0000) 2024-09-10T18:56:31.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[33542]: debug 2024-09-10T18:56:31.617+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:56.334824+0000 front 2024-09-10T18:55:56.334944+0000 (oldest deadline 2024-09-10T18:56:18.034801+0000) 2024-09-10T18:56:31.920 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[33542]: debug 2024-09-10T18:56:31.617+0000 7f3450c14640 -1 osd.5 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:03.837223+0000 front 2024-09-10T18:56:03.837244+0000 (oldest deadline 2024-09-10T18:56:27.237431+0000) 2024-09-10T18:56:31.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[40568]: debug 2024-09-10T18:56:31.589+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:31.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[40568]: debug 2024-09-10T18:56:31.589+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:31.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:31 smithi151 bash[40568]: debug 2024-09-10T18:56:31.589+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:32.589 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:32 smithi151 bash[47658]: debug 2024-09-10T18:56:32.201+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:32.589 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:32 smithi151 bash[47658]: debug 2024-09-10T18:56:32.201+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:32.589 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:32 smithi151 bash[47658]: debug 2024-09-10T18:56:32.201+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:32.875 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:32 smithi151 bash[40568]: debug 2024-09-10T18:56:32.577+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:32.875 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:32 smithi151 bash[40568]: debug 2024-09-10T18:56:32.577+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:32.875 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:32 smithi151 bash[40568]: debug 2024-09-10T18:56:32.577+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:33.503 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[47658]: debug 2024-09-10T18:56:33.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:33.504 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[47658]: debug 2024-09-10T18:56:33.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:33.504 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[47658]: debug 2024-09-10T18:56:33.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:33.504 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[59584]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-5 2024-09-10T18:56:33.919 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[59651]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-5 2024-09-10T18:56:33.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[40568]: debug 2024-09-10T18:56:33.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:33.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[40568]: debug 2024-09-10T18:56:33.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:33.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:33 smithi151 bash[40568]: debug 2024-09-10T18:56:33.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:34.617 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:34 smithi151 bash[47658]: debug 2024-09-10T18:56:34.221+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:34.617 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:34 smithi151 bash[47658]: debug 2024-09-10T18:56:34.221+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:34.617 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:34 smithi151 bash[47658]: debug 2024-09-10T18:56:34.221+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:34.919 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:34 smithi151 bash[40568]: debug 2024-09-10T18:56:34.605+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:34.919 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:34 smithi151 bash[40568]: debug 2024-09-10T18:56:34.605+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:34.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:34 smithi151 bash[40568]: debug 2024-09-10T18:56:34.605+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:35.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[47658]: debug 2024-09-10T18:56:35.185+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:35.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[47658]: debug 2024-09-10T18:56:35.185+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:35.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[47658]: debug 2024-09-10T18:56:35.185+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:35.726 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T18:56:35.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:56:35.758 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T18:56:35.758 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T18:56:35.759 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.6 2024-09-10T18:56:35.919 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[40568]: debug 2024-09-10T18:56:35.653+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:35.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[40568]: debug 2024-09-10T18:56:35.653+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:35.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[40568]: debug 2024-09-10T18:56:35.653+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:35.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:35 smithi151 systemd[1]: Stopping Ceph osd.6 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:56:35.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[40568]: debug 2024-09-10T18:56:35.901+0000 7ff632477640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:56:35.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[40568]: debug 2024-09-10T18:56:35.901+0000 7ff632477640 -1 osd.6 774 *** Got signal Terminated *** 2024-09-10T18:56:35.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:35 smithi151 bash[40568]: debug 2024-09-10T18:56:35.901+0000 7ff632477640 -1 osd.6 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:56:36.623 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:36 smithi151 bash[47658]: debug 2024-09-10T18:56:36.177+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:36.623 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:36 smithi151 bash[47658]: debug 2024-09-10T18:56:36.177+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:36.623 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:36 smithi151 bash[47658]: debug 2024-09-10T18:56:36.177+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:36.623 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:36 smithi151 bash[47658]: debug 2024-09-10T18:56:36.177+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:36.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:36 smithi151 bash[40568]: debug 2024-09-10T18:56:36.613+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:36.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:36 smithi151 bash[40568]: debug 2024-09-10T18:56:36.613+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:36.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:36 smithi151 bash[40568]: debug 2024-09-10T18:56:36.613+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:37.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:37 smithi151 bash[47658]: debug 2024-09-10T18:56:37.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:37.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:37 smithi151 bash[47658]: debug 2024-09-10T18:56:37.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:37.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:37 smithi151 bash[47658]: debug 2024-09-10T18:56:37.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:37.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:37 smithi151 bash[47658]: debug 2024-09-10T18:56:37.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:37.919 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:37 smithi151 bash[40568]: debug 2024-09-10T18:56:37.649+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:37.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:37 smithi151 bash[40568]: debug 2024-09-10T18:56:37.649+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:37.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:37 smithi151 bash[40568]: debug 2024-09-10T18:56:37.649+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:38.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:38 smithi151 bash[47658]: debug 2024-09-10T18:56:38.117+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:38.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:38 smithi151 bash[47658]: debug 2024-09-10T18:56:38.117+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:38.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:38 smithi151 bash[47658]: debug 2024-09-10T18:56:38.117+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:38.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:38 smithi151 bash[47658]: debug 2024-09-10T18:56:38.117+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:38.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:38 smithi151 bash[40568]: debug 2024-09-10T18:56:38.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:38.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:38 smithi151 bash[40568]: debug 2024-09-10T18:56:38.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:38.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:38 smithi151 bash[40568]: debug 2024-09-10T18:56:38.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:39.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[47658]: debug 2024-09-10T18:56:39.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:39.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[47658]: debug 2024-09-10T18:56:39.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:39.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[47658]: debug 2024-09-10T18:56:39.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:39.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[47658]: debug 2024-09-10T18:56:39.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:39.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[40568]: debug 2024-09-10T18:56:39.573+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:39.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[40568]: debug 2024-09-10T18:56:39.573+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:39.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[40568]: debug 2024-09-10T18:56:39.573+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:39.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:39 smithi151 bash[40568]: debug 2024-09-10T18:56:39.573+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:13.123191+0000 front 2024-09-10T18:56:13.123376+0000 (oldest deadline 2024-09-10T18:56:39.023100+0000) 2024-09-10T18:56:40.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[47658]: debug 2024-09-10T18:56:40.133+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:40.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[47658]: debug 2024-09-10T18:56:40.133+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:40.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[47658]: debug 2024-09-10T18:56:40.133+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:40.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[47658]: debug 2024-09-10T18:56:40.133+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:40.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:40 smithi100 bash[68068]: ts=2024-09-10T18:56:40.162Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:40.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:40 smithi100 bash[68068]: ts=2024-09-10T18:56:40.163Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:40.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:40 smithi100 bash[68068]: ts=2024-09-10T18:56:40.164Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:56:40.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[40568]: debug 2024-09-10T18:56:40.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.218343+0000 front 2024-09-10T18:55:48.218133+0000 (oldest deadline 2024-09-10T18:56:14.117727+0000) 2024-09-10T18:56:40.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[40568]: debug 2024-09-10T18:56:40.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:55.819238+0000 front 2024-09-10T18:55:55.819319+0000 (oldest deadline 2024-09-10T18:56:21.718860+0000) 2024-09-10T18:56:40.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[40568]: debug 2024-09-10T18:56:40.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:04.020267+0000 front 2024-09-10T18:56:04.020313+0000 (oldest deadline 2024-09-10T18:56:29.220669+0000) 2024-09-10T18:56:40.920 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:40 smithi151 bash[40568]: debug 2024-09-10T18:56:40.617+0000 7ff62ba89640 -1 osd.6 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:13.123191+0000 front 2024-09-10T18:56:13.123376+0000 (oldest deadline 2024-09-10T18:56:39.023100+0000) 2024-09-10T18:56:41.391 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:41 smithi151 bash[47658]: debug 2024-09-10T18:56:41.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:41.391 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:41 smithi151 bash[47658]: debug 2024-09-10T18:56:41.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:41.391 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:41 smithi151 bash[47658]: debug 2024-09-10T18:56:41.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:41.391 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:41 smithi151 bash[47658]: debug 2024-09-10T18:56:41.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:42.289 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:41 smithi151 bash[59789]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-6 2024-09-10T18:56:42.289 INFO:journalctl@ceph.osd.6.smithi151.stdout:Sep 10 18:56:42 smithi151 bash[59862]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-6 2024-09-10T18:56:42.289 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:42 smithi151 bash[47658]: debug 2024-09-10T18:56:42.137+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:42.290 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:42 smithi151 bash[47658]: debug 2024-09-10T18:56:42.137+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:42.290 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:42 smithi151 bash[47658]: debug 2024-09-10T18:56:42.137+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:42.290 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:42 smithi151 bash[47658]: debug 2024-09-10T18:56:42.137+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:43.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:43 smithi151 bash[47658]: debug 2024-09-10T18:56:43.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:43.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:43 smithi151 bash[47658]: debug 2024-09-10T18:56:43.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:43.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:43 smithi151 bash[47658]: debug 2024-09-10T18:56:43.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:43.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:43 smithi151 bash[47658]: debug 2024-09-10T18:56:43.113+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:44.300 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T18:56:44.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:56:44.333 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T18:56:44.333 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T18:56:44.334 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.7 2024-09-10T18:56:44.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 bash[47658]: debug 2024-09-10T18:56:44.153+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:44.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 bash[47658]: debug 2024-09-10T18:56:44.153+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:44.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 bash[47658]: debug 2024-09-10T18:56:44.153+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:44.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 bash[47658]: debug 2024-09-10T18:56:44.153+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:44.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 systemd[1]: Stopping Ceph osd.7 for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:56:44.919 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 bash[47658]: debug 2024-09-10T18:56:44.489+0000 7f33bd067640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:56:44.920 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 bash[47658]: debug 2024-09-10T18:56:44.489+0000 7f33bd067640 -1 osd.7 774 *** Got signal Terminated *** 2024-09-10T18:56:44.920 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:44 smithi151 bash[47658]: debug 2024-09-10T18:56:44.489+0000 7f33bd067640 -1 osd.7 774 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:56:45.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:45 smithi151 bash[47658]: debug 2024-09-10T18:56:45.121+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:45.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:45 smithi151 bash[47658]: debug 2024-09-10T18:56:45.121+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:45.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:45 smithi151 bash[47658]: debug 2024-09-10T18:56:45.121+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:45.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:45 smithi151 bash[47658]: debug 2024-09-10T18:56:45.121+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:46.419 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:46 smithi151 bash[47658]: debug 2024-09-10T18:56:46.149+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:46.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:46 smithi151 bash[47658]: debug 2024-09-10T18:56:46.149+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:46.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:46 smithi151 bash[47658]: debug 2024-09-10T18:56:46.149+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:46.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:46 smithi151 bash[47658]: debug 2024-09-10T18:56:46.149+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:47.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:47 smithi151 bash[47658]: debug 2024-09-10T18:56:47.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:47.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:47 smithi151 bash[47658]: debug 2024-09-10T18:56:47.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:47.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:47 smithi151 bash[47658]: debug 2024-09-10T18:56:47.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:47.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:47 smithi151 bash[47658]: debug 2024-09-10T18:56:47.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:47.420 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:47 smithi151 bash[47658]: debug 2024-09-10T18:56:47.145+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.151:6802 osd.4 since back 2024-09-10T18:56:22.722805+0000 front 2024-09-10T18:56:22.722986+0000 (oldest deadline 2024-09-10T18:56:46.822729+0000) 2024-09-10T18:56:48.669 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:48 smithi151 bash[47658]: debug 2024-09-10T18:56:48.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:48.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:48 smithi151 bash[47658]: debug 2024-09-10T18:56:48.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:48.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:48 smithi151 bash[47658]: debug 2024-09-10T18:56:48.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:48.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:48 smithi151 bash[47658]: debug 2024-09-10T18:56:48.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:48.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:48 smithi151 bash[47658]: debug 2024-09-10T18:56:48.189+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.151:6802 osd.4 since back 2024-09-10T18:56:22.722805+0000 front 2024-09-10T18:56:22.722986+0000 (oldest deadline 2024-09-10T18:56:46.822729+0000) 2024-09-10T18:56:49.669 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:49 smithi151 bash[47658]: debug 2024-09-10T18:56:49.233+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6803 osd.0 since back 2024-09-10T18:55:48.113565+0000 front 2024-09-10T18:55:48.113635+0000 (oldest deadline 2024-09-10T18:56:09.813465+0000) 2024-09-10T18:56:49.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:49 smithi151 bash[47658]: debug 2024-09-10T18:56:49.233+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6807 osd.1 since back 2024-09-10T18:55:57.217187+0000 front 2024-09-10T18:55:57.217402+0000 (oldest deadline 2024-09-10T18:56:22.517342+0000) 2024-09-10T18:56:49.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:49 smithi151 bash[47658]: debug 2024-09-10T18:56:49.233+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6811 osd.2 since back 2024-09-10T18:56:05.919715+0000 front 2024-09-10T18:56:05.919513+0000 (oldest deadline 2024-09-10T18:56:31.219255+0000) 2024-09-10T18:56:49.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:49 smithi151 bash[47658]: debug 2024-09-10T18:56:49.233+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.100:6815 osd.3 since back 2024-09-10T18:56:12.320655+0000 front 2024-09-10T18:56:12.320553+0000 (oldest deadline 2024-09-10T18:56:35.820315+0000) 2024-09-10T18:56:49.670 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:49 smithi151 bash[47658]: debug 2024-09-10T18:56:49.233+0000 7f33b867d640 -1 osd.7 774 heartbeat_check: no reply from 172.21.15.151:6802 osd.4 since back 2024-09-10T18:56:22.722805+0000 front 2024-09-10T18:56:22.722986+0000 (oldest deadline 2024-09-10T18:56:46.822729+0000) 2024-09-10T18:56:50.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:50 smithi100 bash[68068]: ts=2024-09-10T18:56:50.163Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:50.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:50 smithi100 bash[68068]: ts=2024-09-10T18:56:50.164Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:56:50.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:56:50 smithi100 bash[68068]: ts=2024-09-10T18:56:50.165Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:56:50.819 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:50 smithi151 bash[60003]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-7 2024-09-10T18:56:50.820 INFO:journalctl@ceph.osd.7.smithi151.stdout:Sep 10 18:56:50 smithi151 bash[60077]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-osd-7 2024-09-10T18:56:52.812 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T18:56:52.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:56:52.847 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T18:56:52.847 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T18:56:52.847 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T18:56:53.294 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Sep 10 18:56:52 smithi100 systemd[1]: Stopping Ceph rgw.foo.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:56:53.294 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Sep 10 18:56:53 smithi100 bash[63762]: debug 2024-09-10T18:56:53.034+0000 7f430affd640 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-09-10T18:56:53.294 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Sep 10 18:56:53 smithi100 bash[63762]: debug 2024-09-10T18:56:53.034+0000 7f431bc247c0 -1 shutting down 2024-09-10T18:57:00.544 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:00 smithi100 bash[68068]: ts=2024-09-10T18:57:00.164Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:57:00.545 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:00 smithi100 bash[68068]: ts=2024-09-10T18:57:00.165Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:57:00.545 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:00 smithi100 bash[68068]: ts=2024-09-10T18:57:00.167Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:57:02.043 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:01 smithi100 bash[68068]: ts=2024-09-10T18:57:01.634Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:57:02.044 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:01 smithi100 bash[68068]: ts=2024-09-10T18:57:01.635Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:57:03.720 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Sep 10 18:57:03 smithi100 bash[141172]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T18:57:03.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T18:57:03.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:57:03.773 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T18:57:03.774 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T18:57:03.774 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T18:57:04.170 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 systemd[1]: Stopping Ceph prometheus.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:57:04.171 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:57:04.171 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:57:04.171 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:57:04.171 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:57:04.171 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:57:04.171 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:57:04.172 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:57:04.172 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.939Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:57:04.172 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.940Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:57:04.172 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.940Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:57:04.172 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.940Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:57:04.172 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Sep 10 18:57:03 smithi151 bash[57984]: ts=2024-09-10T18:57:03.941Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:57:04.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T18:57:04.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:57:04.774 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T18:57:04.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:57:10.293 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:10 smithi100 bash[68068]: ts=2024-09-10T18:57:10.165Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:57:10.294 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:10 smithi100 bash[68068]: ts=2024-09-10T18:57:10.166Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:57:10.294 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:10 smithi100 bash[68068]: ts=2024-09-10T18:57:10.167Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:57:11.634 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:57:11 smithi100 systemd[1]: Stopping Ceph node-exporter.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:57:12.043 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:11 smithi100 bash[68068]: ts=2024-09-10T18:57:11.634Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:57:12.044 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:11 smithi100 bash[68068]: ts=2024-09-10T18:57:11.635Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi100.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.100:8443: connect: connection refused" 2024-09-10T18:57:12.044 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:11 smithi100 bash[68068]: ts=2024-09-10T18:57:11.636Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi151.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.151:8443: connect: connection refused" 2024-09-10T18:57:12.488 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:57:12 smithi100 bash[141603]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T18:57:12.488 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:57:12 smithi100 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:57:12.488 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:57:12 smithi100 bash[141677]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T18:57:12.489 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:57:12 smithi100 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T18:57:12.489 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Sep 10 18:57:12 smithi100 systemd[1]: Stopped Ceph node-exporter.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:57:12.793 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:12 smithi100 systemd[1]: Stopping Ceph alertmanager.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:57:12.794 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:12 smithi100 bash[68068]: ts=2024-09-10T18:57:12.633Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:57:13.403 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:13 smithi100 bash[141735]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:57:13.403 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:13 smithi100 bash[141809]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:57:13.403 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:13 smithi100 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@alertmanager.a.service: Succeeded. 2024-09-10T18:57:13.404 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Sep 10 18:57:13 smithi100 systemd[1]: Stopped Ceph alertmanager.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:57:14.101 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:57:20.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:20 smithi151 systemd[1]: Stopping Ceph iscsi.iscsi.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:57:20.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:20 smithi151 bash[54154]: debug Shutdown received 2024-09-10T18:57:31.148 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:30 smithi151 bash[60526]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T18:57:31.419 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:31 smithi151 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T18:57:31.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:31 smithi151 bash[60604]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T18:57:32.097 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:31 smithi151 bash[60612]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T18:57:32.097 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:31 smithi151 bash[60648]: rm: cannot remove '/run/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@iscsi.iscsi.a.tcmu.service-pid': No such file or directory 2024-09-10T18:57:32.097 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:31 smithi151 bash[60649]: rm: cannot remove '/run/ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@iscsi.iscsi.a.tcmu.service-cid': No such file or directory 2024-09-10T18:57:32.098 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:31 smithi151 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T18:57:32.098 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Sep 10 18:57:31 smithi151 systemd[1]: Stopped Ceph iscsi.iscsi.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:57:32.421 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:57:32 smithi151 systemd[1]: Stopping Ceph node-exporter.b for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:57:33.419 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:57:33 smithi151 bash[60731]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T18:57:33.419 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:57:33 smithi151 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:57:33.420 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:57:33 smithi151 bash[60801]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T18:57:33.420 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:57:33 smithi151 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T18:57:33.420 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Sep 10 18:57:33 smithi151 systemd[1]: Stopped Ceph node-exporter.b for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:57:33.798 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:57:33 smithi151 systemd[1]: Stopping Ceph grafana.a for 505ebbde-6fa1-11ef-bcea-c7b262605968... 2024-09-10T18:57:34.048 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:57:34 smithi151 bash[60862]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-grafana-a 2024-09-10T18:57:34.510 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:57:34 smithi151 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:57:34.510 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:57:34 smithi151 bash[60934]: Error response from daemon: No such container: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968-grafana-a 2024-09-10T18:57:34.510 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:57:34 smithi151 systemd[1]: ceph-505ebbde-6fa1-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T18:57:34.510 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Sep 10 18:57:34 smithi151 systemd[1]: Stopped Ceph grafana.a for 505ebbde-6fa1-11ef-bcea-c7b262605968. 2024-09-10T18:57:34.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:35.007 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:35.020 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T18:57:35.021 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216/remote/smithi100/crash 2024-09-10T18:57:35.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:57:35.063 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:57:35.063 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:57:35.065 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216/remote/smithi151/crash 2024-09-10T18:57:35.066 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:57:35.080 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:57:35.081 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:57:35.081 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T18:57:35.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-09-10T18:57:35.098 INFO:teuthology.orchestra.run.smithi100.stderr:grep: /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T18:57:35.100 INFO:tasks.cephadm:Compressing logs... 2024-09-10T18:57:35.100 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:57:35.143 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:57:35.157 INFO:teuthology.orchestra.run.smithi100.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-09-10T18:57:35.159 INFO:teuthology.orchestra.run.smithi151.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-09-10T18:57:35.171 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T18:57:35.171 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T18:57:35.171 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T18:57:35.172 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T18:57:35.172 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T18:57:35.172 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T18:57:35.172 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T18:57:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.1.log: /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:57:35.179 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T18:57:35.182 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T18:57:35.182 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T18:57:35.183 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T18:57:35.183 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T18:57:35.183 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T18:57:35.183 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T18:57:35.183 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/tcmu-runner.log: 63.1% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T18:57:35.183 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:57:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T18:57:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:57:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.7.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:57:35.190 INFO:teuthology.orchestra.run.smithi151.stderr: 92.4% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T18:57:35.191 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:57:35.197 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:57:35.198 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.2.log: 91.1% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:57:35.220 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: 94.1% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T18:57:35.227 INFO:teuthology.orchestra.run.smithi100.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:57:35.614 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T18:57:36.037 INFO:teuthology.orchestra.run.smithi100.stderr: 92.6% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T18:57:36.532 INFO:teuthology.orchestra.run.smithi151.stderr: 92.9% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T18:57:36.998 INFO:teuthology.orchestra.run.smithi100.stderr: 92.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T18:57:40.201 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T18:57:40.322 INFO:teuthology.orchestra.run.smithi151.stderr: 94.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T18:57:40.539 INFO:teuthology.orchestra.run.smithi151.stderr: 94.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T18:57:40.803 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T18:57:40.911 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T18:57:40.937 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T18:57:41.552 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T18:57:41.554 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-10T18:57:41.554 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m6.408s 2024-09-10T18:57:41.554 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m25.810s 2024-09-10T18:57:41.554 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m0.924s 2024-09-10T18:57:41.562 INFO:teuthology.orchestra.run.smithi151.stderr: 94.1% -- replaced with /var/log/ceph/505ebbde-6fa1-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T18:57:41.564 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-10T18:57:41.564 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m6.416s 2024-09-10T18:57:41.564 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m22.574s 2024-09-10T18:57:41.564 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.793s 2024-09-10T18:57:41.565 INFO:tasks.cephadm:Archiving logs... 2024-09-10T18:57:41.565 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216/remote/smithi100/log 2024-09-10T18:57:41.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:57:45.100 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216/remote/smithi151/log 2024-09-10T18:57:45.101 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:57:48.372 INFO:tasks.cephadm:Removing cluster... 2024-09-10T18:57:48.372 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 --force 2024-09-10T18:57:49.965 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 505ebbde-6fa1-11ef-bcea-c7b262605968 --force 2024-09-10T18:57:51.546 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T18:57:51.546 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T18:57:51.553 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T18:57:51.558 INFO:tasks.cephadm:Teardown complete 2024-09-10T18:57:51.558 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T18:57:51.567 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T18:57:51.567 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:57:51.598 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:57:51.662 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T18:57:51.671 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T18:57:51.671 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:57:51.673 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:57:51.694 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T18:57:51.695 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-09-10T18:57:51.695 INFO:teuthology.orchestra.run.smithi151.stdout:*hv01.front.sepi 67.205.162.81 3 u 8 64 377 0.099 8.219 1.240 2024-09-10T18:57:51.695 INFO:teuthology.orchestra.run.smithi151.stdout:+hv02.front.sepi 63.231.80.2 3 u 21 64 377 0.104 6.819 0.997 2024-09-10T18:57:51.695 INFO:teuthology.orchestra.run.smithi151.stdout:+hv03.front.sepi 74.6.168.72 3 u 23 64 377 0.093 7.919 1.064 2024-09-10T18:57:51.695 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-09-10T18:57:51.696 INFO:teuthology.orchestra.run.smithi100.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T18:57:51.696 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================== 2024-09-10T18:57:51.697 INFO:teuthology.orchestra.run.smithi100.stdout:*hv01.front.sepi 67.205.162.81 3 u 5 64 377 0.099 -1.018 0.106 2024-09-10T18:57:51.697 INFO:teuthology.orchestra.run.smithi100.stdout:+hv02.front.sepi 63.231.80.2 3 u 9 64 377 0.091 -1.927 0.097 2024-09-10T18:57:51.697 INFO:teuthology.orchestra.run.smithi100.stdout:+hv03.front.sepi 74.6.168.72 3 u 14 64 377 0.098 0.021 0.166 2024-09-10T18:57:51.697 INFO:teuthology.orchestra.run.smithi100.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-09-10T18:57:51.697 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T18:57:51.705 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T18:57:51.706 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T18:57:51.713 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T18:57:51.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T18:57:51.729 INFO:teuthology.task.internal:Duration was 2718.331139 seconds 2024-09-10T18:57:51.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T18:57:51.737 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T18:57:51.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:57:51.746 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:57:51.798 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T18:57:51.798 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-09-10T18:57:51.798 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:57:51.814 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-09-10T18:57:51.814 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:57:51.826 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T18:57:51.827 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:57:51.829 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:57:51.903 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T18:57:51.903 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:57:51.905 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:57:52.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T18:57:52.097 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T18:57:52.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:57:52.114 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:57:52.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T18:57:52.139 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:57:52.142 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:57:52.154 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-09-10T18:57:52.155 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-09-10T18:57:52.192 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:57:52.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.214 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:57:52.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:57:52.242 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T18:57:52.251 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T18:57:52.252 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216/remote/smithi100 2024-09-10T18:57:52.253 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:57:52.289 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900216/remote/smithi151 2024-09-10T18:57:52.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:57:52.320 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T18:57:52.320 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:57:52.322 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:57:52.363 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T18:57:52.373 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T18:57:52.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T18:57:52.382 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T18:57:52.382 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:57:52.384 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:57:52.389 INFO:teuthology.orchestra.run.smithi100.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:57 /home/ubuntu/cephtest 2024-09-10T18:57:52.392 INFO:teuthology.orchestra.run.smithi151.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:57 /home/ubuntu/cephtest 2024-09-10T18:57:52.394 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T18:57:52.403 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T18:57:52.424 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} duration: 2718.3311393260956 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T18:57:52.424 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:57:52.521 INFO:teuthology.run:pass